Well, the fact that you are new to running is probably the big reason - but you already knew that... But your knee also hurts, which is not always a good thing. There are a lot of things that could be going wrong: running too fast, trying to run too far, running too often, or all of the above and more. I'd recommend that you throttle back a little bit, and at least work on walk-running in order to build out your legs and get them used to the exercise. Careful with your knee though - if it continues to bother you, consider getting medical attention. You mention your shoes - make sure that they are good shoes that fit you properly, i.e. go to a running store and consult with them re a proper shoe for you. If your current shoes are somewhat old, the support that they are supposed to provide you may not be there, perhaps contributing to your knee issues.
Best of luck!
-b