The tub should be supported underneath where it sits on the subfloor, not just at the lip. Usually the manufacturer specifies that the tub be set into a bed of thinset mortar to make sure it's well supported across the entire bottom surface. We have a tub that's doing the same thing.
The easy fix is to remove all the caulk around the edges, then fill the tub with water before recaulking, only draining the tub after the caulk has had time to dry fully. This ensures that the caulk seam stays mostly in compression even when the tub is full of water and people, reducing the chance that it will get pulled apart. I've done this to ours and it's lasted well so far (granted, it's only been a few months now -- we just moved here in April).
What I'd eventually like to do, as our fiberglass tub still creaks when you walk in it, is to open up the adjacent wall and try to get thinset mortar underneath the tub. My plan is to gently pry the base of the tub upward with shims, then slather mortar underneath as much as possible, and then remove the shims so that the tub settles into the mortar. This might be an option for you, but it might involve some tile work depending on where you can get access to the subfloor under the tub.
I know this doesn't do much to answer your specific question about the feasibility of removing and replacing your tub. That's difficult to answer without knowing exactly how your tub is installed. I have to think that's going to be a big job though, no matter how it's installed.
I don't quite follow your description, the standpipe configuration might be part of the problem. The gurgling toilet indicates a venting problem, I don't think it would contribute to overflowing, but it is a problem. As mentioned, a photo or diagram is needed for better assessment. My intent is to describe a proper washer installation and address how the height plays into it.
There are all sorts of variations for pipe configurations, this one is very typical. The washer drain ties into the 4" horizontal pipe with a vertical 4x4x2 LTTY (long turn tee-wye). It can be installed flat to gain some vertical, at the cost of an extra ell that introduces a potential clog point.
From the LTTY is a vertical 2" pipe as short as possible before connecting to a 2" sanitary tee. A 2" vertical pipe continues up to tie into the venting system. The vent could be reduced to 1.5" if need be. The side inlet of the santee takes a 2" horizontal (2% slope actually) trap arm. There is a length limit, but it's rather long, it's best to keep it rather short. The trap arm connects to a 2" trap assembly. The trap inlet accepts a 2" vertical stand pipe.
The stand pipe must be a minimum 18" high, straight up, no bends. You cannot connect a washer discharge directly to a closed drain system. There must be an overflow provision such as an open stand pipe. I prefer to see about a 24" stand pipe, but 18" is a code requirement.
This gives you about a 24-30 inch tall stand pipe/trap assembly on top of the 48" height of the main drain, so a total 6-6.5 feet off the floor. Washers have a limit of how high they can pump discharge water. The installation instructions should tell you this somewhere. If not, you may need to contact the manufacturer. If your washer can pump up to the top of the stand pipe while sitting on the floor, you're all set. If not, you will need a platform that raises the washer to within the allowable discharge height.
If you still get overflowing with a proper standpipe-trap configuration, then the main line remains obstructed somewhere.
Best Answer
For common bathtubs the correct overflow gasket should be wedge shaped with a flat side and an angled side. The flat side mates with the overflow horn (the piece directly behind the hole) and the angled side mates with the backside of the tub, fat part down to match the slope of the tub wall.
I have seen them made of both solid rubber and foam rubber. I prefer the foam rubber because you can get a decent squish on them and they will conform to irregularities on the back side of cast iron tubs (sometimes the porcelain glaze is slopped a bit back there, sometimes the casting is rough).
Do not attempt to smear caulk around it, it won't work. They can be replaced without ripping out the tub. Remove the overflow plate and take a look, you should be able to pull out the old gasket with a combination of long tools and needle-nose pliers (I use a hemostat- see picture).
There is always a bit of "give" on the overflow pipe, which helps, but try not to move it too much or you risk compromising the connection to the waste tee below (if it is the type with a slip-joint connection, many are glued plastic).
Get a piece of slim stiff wire about 8" long and bent in a "j" shape on one end and keep it handy. Get a helper too, unless you have 3 arms and hands. My technique for this is to install a plate screw into the horn so you have something to grab to apply pressure to the new gasket as you slide it into place (the screw will also prevent the new gasket from falling behind the tub if you drop it). Manipulate the gasket with your free hand, using tools if you have to, until it is positioned properly (you may have to jam it in a bit). When the gasket is in the right position pull back on the screw to hold the gasket in position between the horn and tub wall. Now use the wire to grip the bottom edge of the shoe to maintain pressure, have your helper hold it. Remove the screw and install the overflow plate (the wire should lay where the overflow ports on the bottom of the plate are).
Unhook the wire from below and remove, and you are done. Be aware that the seal provided by the gasket is not really meant to be totally watertight like the drain is, it's not meant to be submerged, it's there for incidental overfill protection.