Solidity Libraries – Understanding Solidity Libraries

I think I have a use case for a library, but I've yet to use one.

I want to deploy a Manager contract as a token, and another for a crowdfund, that will each hold all their respective storage variables, and simply delegate every function call down to libraries that perform the actual logic. In this way I can update bugs if they are found by re-uploading the Library and changing the Library address, and all of my storage will persist.

If I have a balances mapping in my ERC20 Manager contract, can the Library access that mapping? Do I need to pass it in as a parameter, or can I just pass in the parameters that I received in the Contract call? Do I even need to do that, since it is run in the context of the calling contract? In the library can I simply go balances[address] += whatever in my transfer function and because I will have a balances mapping on my contract, it will work and find the address in that mapping, and if I didn't have that mapping on my contract, the library function would just throw?

I guess the root of what I want is, to store no variables/struct/anything in the Library, just a list of functions, so I can simply replace the business logic of my app, kind of creating a server/database sort of thing. But is the library aware of the variables on my calling contract automatically?

EDIT: After more research, it seems you can not update a reference to a library once compiled. So, I would instead create a Contract that does what my Library does, no storage, just functions, and run delegatecalls to it. My question stands – does the contract have access to all of the storage of the calling contract automatically?