Wiring colour conventions can vary worldwide, so as you don't state where you are located, it's not possible to give a definitive answer.
In the UK for example (for single phase wiring), convention would have the live wire as brown (new colour) or red (old colour) and the neutral wire as blue (new colour) or black (old colour). It was not uncommon to see appliances using the new colours while fixed wiring still used the old colours, until the new colour scheme was adopted for fixed wiring (around 2004).
If you are in any doubt at all, sound advice would be to contact a competent electrician to do the work for you.
In a perfect world, with perfect electronics, there would be no need for a grounding conductor. It's there to deal with imperfections. The most likely case for a ground fault in this installation would be if the black conductor lost some insulation and shorted itself against the frame of the fan. In such a situation, the casing could become electrified, and a grounded person, touching the fan body could receive a shock.
If it were properly grounded, you'd get a short circuit to the ground wire, and the breaker would trip.
However, the fan is mounted to the ceiling, and the possibility of someone touching it is low. (Unlike, for example, the frame of a fridge or stove, where human contact is part of normal operation.)
So, Is it safe? Quite likely. For added safety, turn the wall switch off when changing bulbs. Is it to code? No. -- Unless, as others have suggested in comments, the mounting box is grounded on the other side.
You can however, locate the breaker, and replace it with a Ground Fault Interrupt breaker, which would trip should a ground fault occur. This isn't something you should do yourself unless you have experience working in the panel.
Another possibility, if your switch happens to be chained off of an outlet, which is quite possible, depending on the routing of the wiring on that circuit, is to replace the outlet with a GFCI outlet, and wire the switch to the load side of it.
Best Answer
Ok, so you will want to connect the white wires from the yellow, red, green, and the fixture. Connect all the black wires together except the fixture.
Now the only two wires you have left are the white wire from the switch and the black wire of the fixture. The white wire will be the "switched hot" and needs to be identified by wrapping a strip of black tape around it. You don't need to cover up all the white, you just need to have an obvious band of black to mark that wire as hot.
Connect that switched hot white wire to the black wire of the fixture and everything should work fine. Note that the larger bundles of wires will need to use larger wire nuts (typically red). Don't try to force more wires into a small yellow wire nut.
Also, I hope the zip ties in the back of that box are not holding the box in place. The box shouldn't wiggle and should be very securely mounted with strong screws if it's going to hold a fan.