It depends what part of the html you what to change, where you should do it. There are two places to do this, either the templates used, where you can create your own with the markup as you please. You could also overwrite the theme functions that panel use and create your custom markup there.
Edit:
The templates are located at panels/plugins/layouts. These are the templates for the page layout, printing the different regions on a panel. The other templates that are used comes from different modules depending on what you put into the panel page. So views in a panel will use the views templates ect. However this is also printed through the panel theme functions, theme_panel_pane in particular if I remember correctly. So there are some possibilities, but again, where to do your overwrite depends on what exactly you want to change.