I'm not familiar with WordPress per se, so I'll just try to point you in the right direction until someone comes along who can answer more specifically.
This is typically handled in CSS. I'd imagine there's some external stylesheet like Style.css or Theme.css that provides styling for your app. This may be found in the root of the app, a folder named Styles or something similar, or in a subfolder structure representing the theme(s) you are using.
list-style: none;
will remove bullets altogether
list-style-type: square;
will make bullets square
list-style-image:url("/images/image.gif");
will use an image you provide.
Do the php files specify a style class for these elements? If not, the stylesheet may be using their ID.
If you can find the stylesheet(s) for your site, find or create the entries for the lists in question.
For example: ul.sidebar1
or #sidebar1 ul
(something along those lines,) and modify the rules to your liking.