Well, first language is important.
You want a computer, minus the peripherals. A computer is made up of different components, and the hard drive is one of them. So is the CPU. There was a time when people would point to the desktop, call that a CPU, a modem, a hard drive incorrectly. Or people like to confuse storage and memory, which is also a big issue today. But that was confusing for a number of reasons, and thanks to sneaky ads that would purposely try to confuse consumers who didn't understand the difference. I think you never self-corrected, making this even harder than it needs to be. But no worries. Aside from what might become a long post.... it's not that complicated. Trust me, read this through, slowly if you need to. It's well worth it.
Okay, so you have 5 basic components that are required for a standard personal computer(sometimes a few of these are combined, but nevertheless they still exist).
-A mainboard or motherboard(let's not get into the differences between the two), 50% of motherboards are synonymous with mainboards with small differences. So you see these two thrown around a bit. Like automobile engines and motors, different things, but in many cases they are synonymous. It's the circuit board, very self-explanatory.
-A processor. This is the CPU or APU(a CPU with major added features). This, like everything else, is connected to the board. It's separate from the board, because the processors come in all types of specifications and generations.
-Memory. This is what the *processor* needs to load data in and out as it is computing and complying with the software you want to run. RAM is memory(Random Access Memory).
-Storage Drive. This is what the *user* needs to store data how they see fit, including the operating system. A Hard Drive is a storage device.
-Power supply. A separate circuit designed to send a clean electrical signal to components in the system. It basically converts the power from your wall/battery, to meet the standards and the power drain requirements of multiple devices in the system itself. Different components require different power draws and the Power supply(or PSU) simplifies all of this, because the components on a board and the devices would take up much more space and cost far more to produce if it was all built in. And power creates heat, which would be another issue. Thus the box inside a box design has been the industry standard for nearly as long as electricity has been used.
There are more components of course. But these are the 5 basic ones every software driven personal computer will have, including mobile devices like smartphones(although they have a different method for a power supply but it still has one, and use another set and style of memory for storage).
From that point, you shop for your needs. You decide what features you need, and the minimum amount of performance you want. Build(either personally DIY, or simply as a consumer looking at specifications on prebuilt solutions) from there. It's almost 2015, so at least understand these 5 components. Understand the basic nomenclatures. It will only make communication easier, as well as shopping easier. Otherwise someone is liable to read your OP, and tell you to buy something that you wouldn't be able to use whatsoever without another big investment.