Balance bikes and the first pedal bike
A balance bike is simply a small bike with no pedals. Children push along with their feet, learn to glide, and pick up the instinct for steering and leaning into corners before they ever have to worry about pedalling. Balance bikes are cheap to buy used, hold up to plenty of abuse, and are the single best investment a parent can make for getting a young child riding well. Most children who ride a balance bike from two years old are riding a pedal bike without stabilisers by four.
The first pedal bike, usually a 14 or 16 inch wheel, should be as light as you can reasonably afford. A cheap supermarket bike at this size weighs close to the child riding it, which makes learning to pedal uphill or turn a frustration. A lightweight alloy bike from Frog, Isla, Hoy, Squish, or the better Specialized, Trek or Giant kids ranges weighs half as much and the difference is enormous for small legs.
Training wheels have gone out of fashion for good reason. A child who rides a balance bike does not need them. If the bike you are buying comes with stabilisers fitted, you can usually remove them and find that the child rides without them inside an afternoon.
Buying used, where the savings really are
The best value in kids' bikes is buying used from parents whose child has outgrown a bike that was barely ridden. A decent child's bike typically gets used intensely for one to two years before the child grows out of it. Frames are usually in excellent condition. Components may need a quick service but nothing is usually worn out.