You always need to build buildings next to an existing urban district. Build something on the farm to the left of where you want to build the library (you'll be able to re-assign the citizen that now works the farm to a new tile), and then you can build the library there.