The issues with the compression fitting have been addressed.
The brass flange fitting, the rubber gasket, and the threaded pipe is the "toilet spud". Replacements are available if the rubber is perished or the metal parts somehow damaged.
The flange nut looks pretty well bonded to the threaded portion. Soak well with WD-40 and let it soak in. There are splines inside the spud on which one would apply an internal spud wrench to keep the threaded pipe part from turning. You may find some other chunk of metal that will wedge against the splines, which in turn can be held by a conventional wrench. Holding the spud in place, remove the flange nut with a suitable tool.
Once the flange nut is removed, push the threaded pipe part inwards, but ensure it will not fall into the toilet. The part inside the toilet is conical shaped. Pushing it inwards will give enough free play that the rubber gasket can be pulled out. Once the rubber is out, there will be enough clearance in the toilet hole that the threaded pipe part can now be removed.
Examine the parts and evaluate if replacement is necessary. If the rubber is not too far gone, it may be possible to clean it, treat it with some sort of rubber treatment, and reinstall. If the rubber is cracked, hardened, rotted, or brittle, it will need to be replaced. The nut should be tightened firmly enough to ensure the rubber seals well against the toilet, but not so tight the rubber gets pushed out of position or the toilet gets cracked or broken.
The drain line is probably filled with hair and soap, chemical drain cleaners at best push the clog further down and make things worse in my opinion (now toilet backing up) with the exception of “mule kick” that will eat metal pipes and I am not sure if it is even available any longer. What you need is a real snake a small one to be run down the shower, and possibly a larger power one to be run down the toilet. Without cleaning the clog this will get worse this kind of snake you hold in one hand and spin pushing it down the drain a little at a time probably less than 2 or 3 chemical treatments. You should have the land lord take care of this incase any damaged pipe or other problems come up and you are not taking the $ hit for something that should be fixed by them.
Best Answer
That is a Kohler "canister style" flush valve.
First, carefully remove the "rim-clean/bowl-fill" water supply hose (the little black hose that runs from your fill valve to the flush valve. Also, remove the chain that attaches the trip lever to the flush valve.
To get at the gasket in question (which is very likely, but not 100% certainly, your problem) you need to carefully twist the "small part above the white ring" counter-clockwise 1/4 turn. This will release the entire canister assembly from the base. It will look like this (note chain still attached in my pic):
The pink washer on the bottom just stretches and pulls on/off. Reverse previous step to reinstall...
NOTE- I have performed this job without removing the hose and chain, but it is easier with them removed.
screen-shot courtesy of REVIEWS@ca.rr.com