You may be confusing two different ways to import code.  You can do it this way:
user> (use 'clojure.zip)
Or while you're declaring a namespace in a source file:
(ns foo
  (:use clojure.zip))
The second version is a macro that is expanded into the first.  
Outside of (ns), doing (:use "zip") is going to treat :use as a function and call it with "zip" as its parameter (i.e. try to use the string "zip" as a collection and look up the key :use in it), which does nothing.
clojure.zip has some functions whose names clash with things in clojure.core though, so you either have to do something like this:
user> (use '(clojure [zip :rename {next next-zip replace replace-zip remove remove-zip}]))
Or preferably this:
user> (require '(clojure [zip :as zip]))
With the latter you can refer to functions like (zip/vector-zip data) as you wish.
See the documentation for require and refer and the page talking about libs.