Slight update. Hooking does work for fraps, but ONLY in Fullscreen mode, which itself is too buggy to work (constant flickering). I'll keep experimenting.
Interestingly, on the flickering side of things, DXtory sees the hook and loses it rapidly.
For those who want to experiment with me, Zelda Classic uses DirectX 7. That may be helpful. Currently I'm trying to force the fullscreen version into a window with dxwnd.
edit: With DXGL, I can manage to get Fraps to record. Still no progress on DXtory. Sadly Fraps doesn't do what I need it to, but this may help others.