Sorry to disappoint, but the headphone jack on the controller is for voice communications only. Game sounds, music, etc. are all played through the display device (tv/monitor).
To get the audio from the Xbox to your headset you'll need some sort of converter cable/connector that turns the red/white connector that goes to your tv to the jack that you have on your headset.
Based on my knowledge there are two solutions for your problem. To get things straight: The original Xbox-pads won't work out of the box with your PS3 and no solution will provide 100% of the features your dualshock-pads have (Sixaxxis, Trigger etc.).
1. Use an Adapter
There are some Adapters on the market (like the XCM which Blem mentioned) which may be pretty pricy, but let you use your Xbox-pads on your PS3. The XCM does not allow you to use sixaxxis and I don't know any adapter which has this feature. Maybe someone who is using this (or another) adapter and can provide more precise information.
2. Use an cheaper 3rd party gamepad
A second way to play on your PS3 with an Xbox-like pad is to buy 3rd partie hardware which can be used on your PS3. I bought the Speedlink XEOX pad, which can be used on the PC and the PS3. The Deadzones of the sticks are not as perfect as they are on the regular dualshocks, but you'll get used to it. Besides that the pad is wireless, so you don't have any wires through your livingroom (the XCM Adapter only works for the wired pad!). There is one little catch with the speedlink-pad with it's triggerbuttons:
Why can’t I operate the trigger buttons (R2 and L2) at the same time?
These two buttons’ signals are on the same axis. As such, they cancel each other out when both buttons are pressed at the same time. This gives you optimal control in many games. Alternatively, select the button assignment in the game so that both trigger buttons don’t need to be pressed at the same time.
Overall this specific pad is not as good as one of the original pads (Xbox / PS3), but is a good and cheap way to play with a Xbox-like pad on your PS3 (or PC).
Best Answer
Yes, provided the videos are encoded in a format that the Xbox360 supports. Also, you'll need to install some software on your MacBook to serve the video. Connect360 and Rivet are commercial products do this. There's also a question over on Super User with some other options.