Whether you enter them via &#xNNNN;
character reference or just by typing or copy-pasting them directly into the page, it's purely a question of fonts. If the user has a glyph for the character in the fonts their web browser knows about, it'll work. If they don't, it'll be a little box.
fileformat.info have font-availability details that can be of help for guessing how widespread glyph support is. eg. hit the Fonts link on this page for U+25B6. For me, this renders on most platforms but not on a vanilla XP install with IE6 (IE7 picks it up from Lucida Sans Unicode which IE6 FSR doesn't bother look at).
◄ ► U+25C4, U+25BA Left/Right-Pointing Pointer glyphs seem to be more widely available than
◀ ▶ U+25C0, U+25B6 Left/Right-Pointing Triangle glyphs
so I'd use those instead. In particlar they render in XP+IE6 with default fonts because they are present in Arial. Which renders them more like Triangle than Pointer is supposed to be. (sigh) OTOH its up/down Triangles are fine.
SO uses ► for list bullets in the ‘how to format’ box over there. ⇗
[response to comment in edit instead of comment as site JS appears to be playing up]
There's also ‣, U+2023 Triangular Bullet, which is in a block of widely-available typographical-niceties. Unfortunately, again, it's not in Arial or other fonts IE6 on a default XP will use. Neither is ▸ U+25B8 Black Right-Pointing Small Triangle.
French-style quotes (guillemets) ‹ › « » will be available absolutely everywhere, but using them for arrows might be considered a bit of an abuse.