Is moving small amount of Bitcoin (0.003 BTC) from exchange to cold wallet worth it?

I want to practice self custody, but I wonder if it's worth moving only a small amount of Bitcoin.

I'm afraid if Bitcoin's suddenly goes up to $150.000 which also causes the transaction fee to tank. If that's the case, I'll lose a lot of money on fees.