Yes, you can put a link on your VF page using the target app's URI scheme. For example, skype://call?415-555-1212 will open the Skype app and start a call.
Not every app has a URI scheme exposed, and those that do may not accept arguments. All of this depends very much on which app you're trying to open.
However, because you're running this in a VF page inside SF1, you won't have access to the native device APIs that would let you detect if the target app is even installed in the first place. If the user doesn't have the target app installed and taps your link, nothing will happen.
If you're on iOS, check out the UIApplication documentation for more details. I've also written a neat little library (SSAppURLs!) that does a lot of the heavy lifting for you.
Lets say you want just mobile look and feel then the approach would be to just style your existing page with CSS friendlier to mobile.You may use plugins like jquery mobile ,polymer,bootsrap .
If you want a responsive UI with better perfomance then would redesign controller to use VF remoting or javascript remoting.This will remove viewstate and make your transaction light .You may consider using frameworks like angular , backbone or knockout to manipulate DOM
Best Answer
The best place to start with this is the Salesforce1 style guide at:
http://sfdc-styleguide.herokuapp.com/
This details colours, fonts and has sample code to create components that match Salesforce1, such as buttons, cards, lists.