Most cameras have built-in memory these days, and that includes the Stylus 1030SW. It has 14.7MB of onboard memory, and if that doesn’t sound like much, you’re right — it holds just three photos at the highest quality setting. Thus, you’ll want to buy a memory card right away, and I’d suggest a 2GB card to start with.
The Stylus natively supports xD Picture Cards, and it can also use the tiny microSD/microSDHC cards via an included adapter. The advantage to using the microSD card is that this format supports much higher capacities: they currently top out at 8GB, compared to 2GB with xD. Do note that you may need to update your camera’s firmware in order to use high capacity microSD cards. Oh, and if you choose to buy an xD card, make sure it’s a "Type M+" (high speed) model.
Digital Camera Resource Page Review: Olympus Stylus 1030SW