The kind of formatting information you want to save can be found in the various properties of the DataGridGiew. You may have to spend some quality time with its object model documentation, but you should be able to find most of what you're looking for.
For example, to save the way the rows have been sorted, you can persist these two properties:
DataGridView.SortedColumn tells you on which column's values the rows were sorted.
DataGridView.SortOrder tells you the order (i.e. ascending or descending)
And you should be able to find info about column widths and order on the members of the DataGridView.Columns collection.
I think it's a matter of preference whether you record these values all at once (e.g. click a "save settings" button) or one by one in event handlers as the properties get changed. I like to save them automatically when the form is closed w/out any action by the user. It's been a while since I implemented anything like this, but I recall the app settings being a pretty handy place to keep this sort of info. In the Project Properties Settings designer, just define settings for the stuff you want to keep (make sure to set the Scope to "user"). Then assign and load actual values in your code at run-time, like so:
// e.g. you defined an integer user setting at design time called ColA_Width, to
// hold the value of the width of "ColA" in your DataGridView.
// Use this code to save the value (either in a specific event handler, or a
// global "save settings" routine).
Properties.Settings.Default.ColA_Width = MyDataGridViewInstance.Columns["ColA"].Width;
Properties.SEttings.Default.Save();
// Then you'd reverse the assignments when the app next loads so that the saved
// settings are "remembered" between user sessions.
MyDataGridViewInstance.Columns["ColA"].Width = Properties.Settings.Default.ColA_Width;
The only caveats I can recall are that some of the DataGridView property values are not straightforward to save as strings (remember that your user settings are ultimately saved as XML), but it really depends on what sort of settings you're saving.