In 1940 the Germans really didn't have much if any experience in using torpedo bombers furthermore they didn't have a good aerial torpedo. I'm not sure if they were using the Italian ones by that point but wouldn't think they had time to get large numbers of them. Certainly German aircraft would have been a threat to RN ships during that period (although night ops by the RN would negate a lot of that) but not as severe as some would think. Several of the early war losses even of ships at speed were impacted by either limited maneuverability or lack of AA ammo as well. Then there's the fact that the LW was rather overtasked with roles for Sea Lion.