Essentially an operating system's windowing system exposes some api calls that you can performm to do jobs like create a window, or put a button on the window. Basically you get a suite of header files and you can call functions in those imported libraries, just like you'd do with stdlib and printf.
Each operating system comes with it's own gui toolkit, suite of header files, and api calls, and their own way of doing things. There are also cross platform toolkits like gtk, qt, and wx widgets that help you build programs that work anywhere. They achieve this by having the same api calls on each platform, but a different implementation for those api functions that call down to the native OS api calls.
One thing they'll all have in common, which will be different from a CLI program, is something called an event loop. The basic idea there is somewhat complicated, and difficult to compress, but in essense it means that not a hell of a lot is going in in your main class/main function, except:
check the event queue if there's any new events
if there is, dispatch those events to appropriate handlers
when you're done, yield control back to the operating system (usually with some kind of special "sleep" or "select" or "yield" function call)
then the yield function will return when the operating system is done, and you have another go around the loop.
There are plenty of resources about event based programming. If you have any experience with javascript, it's the same basic idea, except that you, the scripter have no access or control over the event loop itself, or what events there are, your only job is to write and register handlers.
edit: Oh also, I forgot to mention that gui programming is incredibly complicated and difficult, in general. If you have the option, it's actually much easier to just integrate an embedded webserver into your program and have an html/web based interface. The one exception that i've encountered is Apple's cocoa+xcode+interface builder + tutorials that make it easily the most approachable environment for people new to gui programming that I've seen.