The having clause doesn't mean the same thing as the where clause : when running a simple query, you should use where -- which is what you did in your second query, that works.
having is used when the condition has to be applied on the result of a group by clause.
Which means that, here, your query should be build this way :
$query = "SELECT zips.*
FROM zips
where zips.city LIKE '%$city%'
AND zips.stateabbr LIKE '%$state%'
LIMIT 1";
With that, if you still have an error about a non-existing or not-found column (at least for city and/or stateabbr), it'll be because that column doesn't exist in your table.
In this case, there is not much we can do : you'll have to check the structure of your table, to determine which columns it contains.
You can check that structure using a web-based tool like phpMyAdmin, or using an SQL instruction such as :
desc zips;
For reference, quoting MySQL's manual page for select :
The SQL standard requires that HAVING
must reference only columns in the
GROUP BY clause or columns used in
aggregate functions.
...
Do not use HAVING for items that
should be in the WHERE clause.
For example, do not write the
following:
SELECT col_name FROM tbl_name HAVING col_name > 0;
Write this instead:
SELECT col_name FROM tbl_name WHERE col_name > 0;
...
The HAVING clause can refer to
aggregate functions, which the WHERE
clause cannot