Problem with Scala Dispatch Databinder library

Question

I am using the Dispatch Databinder library for Http in Scala.

I have this method.

def testCheckPage(url:String):String = {
  try {  
    var http = new Http
    var request = new Request(url)                                      
    val req_with_agent = request <:< Map("User-Agent" -> "Mozilla/4.0")  
    val responseBody = Http (req_with_agent as_str)
    responseBody.length.toString
  } catch {
    case ex: Exception if ex.getMessage == null => "Message: " + ex.toString
    case ex: Exception => "Message: " + ex.getMessage 
  }
}

With certain urls, such as http://bae.cf.huffingtonpost.com, I am getting a java.nio.BufferUnderflowException. Any idea whay?

Answer 1

This can occur in other libraries (e.g. Source.fromUrl) when the encoding is mismatched. I'm not sure how to set the encoding in that library, but the encoding for the URL you provided is ISO-8859-1 and I believe this would default to UTF-8, causing that problem.