That is true of APCR but not of APDS. The surrounding sabot (sleeve) is discarded at the muzzle, leaving just the very heavy, small-diameter core to fly on. This is ballistically better than a conventional full-calibre AP shot as it more dense, and it loses velocity quite slowly. The range of the WW2-era APDS wasn't limited by ballistics but by accuracy; dispersion was initially much higher than full-calibre AP, so it was recommended for use at shorter ranges as it would probably miss the target at long range. Tony Williams: Military gun and ammunition website and discussion forum
The picture below (form the Ammo Photo Gallery on my website) shows some sub-calibre AP ammo from my collection. In this pic, the following are APCR: 20x138B (German); 37x94R (French, steel-cored in magnesium sleeve); 37x249R (German); 45x310R (Russian); 50x289R (German) and 57x480R (Russian). The following are squeezebore: 28x187R (German) and 40x304R (British 2 pdr Littlejohn Mk 1). The 57x441R is the British 6 pdr APDS. The next picture shows some of the projectiles by themselves: The 28/20 Gerlich squeezebore has been sectioned to show the tungsten core inside the sleeve. You can clearly see the flanges which are squeezed flat against the core. On the right is the 17 pdr APDS round complete, with its penetrating tungsten core next to it. Tony Williams: Military gun and ammunition website and discussion forum
HESH was developed in WW2. I think that the Burney RCL guns have already been mentioned: Burney is actually given the credit for inventing HESH for these weapons, during the war. There was also a small British anti-tank bomb developed in 1943, as described in 'Flying Guns – World War 2: Development of Aircraft Guns, Ammunition and Installations 1933-45' by Emmanuel Gustin and myself: "The 9 lb AT bomb, jovially known as "Puffball", used a squash-head rather than a HEAT design and a fighter-bomber was expected to carry twenty-four of them, to be released in one diving pass at low altitude. Despite the success of similar (but smaller) Soviet and German weapons, Puffball proved unsatisfactory due to sympathetic detonations in mid-air (the explosion of the first hits setting off the others) and significant damage from blast and debris being suffered by the carrying aircraft." However, I don't know of any HESH warhead which entered service until after WW2. Tony Williams: Military gun and ammunition website and discussion forum