7

Happy new years, first of all!

I'm having some problem parsing JSON in Play, the format I'm dealing with is as follows:

JSON Response:

 ...
"image":{  
    "large":{  
      "path":"http://url.jpg",
      "width":300,
      "height":200
    },
    "medium":{  
      "path":"http://url.jpg",
      "width":200,
      "height":133
    },
    ...
 }
...

I'm stuck on the field with the sizes. They are obviously variables, and I'm not sure how to write a formatter for this? The JSON is coming from an external service.

So far I have

final case class Foo(
  ..
  ..
  image: Option[Image])


final case class Image(size: List[Size])

final case class Size(path: String, width: Int, height: Int)

For the formatting I just did Json.reads[x] for all the classes. However I'm pretty sure that the variable for size is throwing off the formatting because it's not able to create an Image object from the JSON coming in.

Improve this question
7
  • Not seeing a way you can use the reads macro to do this. Do you want to keep the size descriptors? (large, medium..) Commented Jan 1, 2015 at 17:23
  • I know that there are 11 size descriptors and their values. Is there a way to put them in some Seq(size1, size2, ..., sizeN) -> imageDetails or something similar? Commented Jan 1, 2015 at 17:25
  • What about adding the descriptor to the Size class? Size(name: String, path: String, width: Int, height: Int), and then read them into a list? Image(..., sizes: List[Size]) Commented Jan 1, 2015 at 17:29
  • I don't think that would work because in the JSON the size description is not in the object. It's sizeDescription: { path: "", width: "", height: "" } Commented Jan 1, 2015 at 17:33
  • I'm asking if that's an acceptable structure to convert it to. If so, I can make a more detailed answer. It's definitely possible to read it that way. Commented Jan 1, 2015 at 17:36

3 Answers 3

Reset to default
8

Update 2016-07-28

The solution described below breaks Referential Transparency because of the use of the return keyword and is not something I would recommend today. Nevertheless, I am not leaving it as it for historical reasons.

Intro

The issue here is that you need to find someplace to save the key for each Size object in the Image object. There are two ways to do this, one is to save it in the Size object itself. This makes sense because the name is intimately related to the Size object, and it is convenient to store it there. So lets explore that solution first.

A quick note on Symmetry

Before we dive into any solutions, let me first introduce the concept of symmetry. This the idea that when you read any Json value, you can use your Scala model representation to go back to exactly the same Json value.

Symmetry when dealing with marshalled data is not strictly required, indeed sometimes it is either not possible, or enforcing it would be too costly without any real gain. But usually it is fairly easy to achieve and it makes working with the serialization implementation much nicer. In many cases it is required as well.

Save name in Size

import play.api.libs.json.Format
import play.api.libs.json.JsPath
import play.api.libs.json.Reads
import play.api.libs.json.JsValue
import play.api.libs.json.JsResult
import play.api.libs.json.JsSuccess
import play.api.libs.json.JsError
import play.api.libs.json.JsObject
import play.api.libs.json.Json

final case class Foo(images: Option[Image])

object Foo {
  implicit val fooFormat: Format[Foo] = Json.format[Foo]
}

final case class Image(sizes: Seq[Size])

object Image {

  implicit val imagesFormat: Format[Image] =
    new Format[Image] {

      /** @inheritdoc */
      override def reads(json: JsValue): JsResult[Image] = json match {
        case j: JsObject => {
          JsSuccess(Image(j.fields.map{
            case (name, size: JsObject) =>
              if(size.keys.size == 3){
                val valueMap = size.value
                valueMap.get("path").flatMap(_.asOpt[String]).flatMap(
                  p=> valueMap.get("height").flatMap(_.asOpt[Int]).flatMap(
                    h => valueMap.get("width").flatMap(_.asOpt[Int]).flatMap(
                      w => Some(Size(name, p, h, w))
                    ))) match {
                  case Some(value) => value
                  case None => return JsError("Invalid input")
                }
              } else {
                  return JsError("Invalid keys on object")
              }
            case _ =>
              return JsError("Invalid JSON Type")
          }))
        }
        case _ => JsError("Invalid Image")
      }

      /** @inheritdoc */
      override def writes(o: Image): JsValue = {
        JsObject(o.sizes.map((s: Size) =>
          (s.name ->
            Json.obj(
              ("path" -> s.path),
              ("height" -> s.height),
              ("width" -> s.width)))))
      }
    }

}

final case class Size(name: String, path: String, height: Int, width: Int)

In this solution Size does not have any Json serialization or deserialization directly, rather it comes as a product of the Image object. This is because, in order to have symmetric serialization of your Image object you need to keep not only the parameters of the Size object, path, height, and width, but also the name of the Size as specified as the keys on the Image object. If you don't store this you can't go back and forth freely.

So this works as we can see below,

scala> import play.api.libs.json.Json
import play.api.libs.json.Json

scala> Json.parse("""
     | {  
     |     "large":{  
     |       "path":"http://url.jpg",
     |       "width":300,
     |       "height":200
     |     },
     |     "medium":{  
     |       "path":"http://url.jpg",
     |       "width":200,
     |       "height":133
     |     }
     | }""")
res0: play.api.libs.json.JsValue = {"large":{"path":"http://url.jpg","width":300,"height":200},"medium":{"path":"http://url.jpg","width":200,"height":133}}

scala> res0.validate[Image]
res1: play.api.libs.json.JsResult[Image] = JsSuccess(Image(ListBuffer(Size(large,http://url.jpg,200,300), Size(medium,http://url.jpg,133,200))),)

scala>

And very importantly it is both safe and symmetric

scala> Json.toJson(res0.validate[Image].get)
res4: play.api.libs.json.JsValue = {"large":{"path":"http://url.jpg","height":200,"width":300},"medium":{"path":"http://url.jpg","height":133,"width":200}}

scala>

A quick note on Safety

In production code, you never, never, never want to use the .as[T] method on a JsValue. This is because if the data is not what you expected, it blows up without any meaningful error handling. If you must, use .asOpt[T], but a much better choice in general is .validate[T], as this will produce some form of error on failure that you can log and then report back to the user.

Probably a Better Solution

Now, probably a better way to do accomplish this would be to change the Image case class declaration to the following

final case class Image(s: Seq[(String, Size)])

and then keep Size as you originally had it,

final case class Size(path: String, height: Int, width: Int)

Then you merely need to do the following to be safe and symmetric.

If we do this then the implementation becomes much nicer, while still being safe and symmetric.

import play.api.libs.json.Format
import play.api.libs.json.JsPath
import play.api.libs.json.Reads
import play.api.libs.json.JsValue
import play.api.libs.json.JsResult
import play.api.libs.json.JsSuccess
import play.api.libs.json.JsError
import play.api.libs.json.JsObject
import play.api.libs.json.Json

final case class Foo(images: Option[Image])

object Foo {
  implicit val fooFormat: Format[Foo] = Json.format[Foo]
}

final case class Image(sizes: Seq[(String, Size)])

object Image {

  implicit val imagesFormat: Format[Image] =
    new Format[Image] {

      /** @inheritdoc */
      override def reads(json: JsValue): JsResult[Image] = json match {
        case j: JsObject =>
          JsSuccess(Image(j.fields.map{
            case (name, size) =>
              size.validate[Size] match {
                case JsSuccess(validSize, _) => (name, validSize)
                case e: JsError => return e
              }
          }))
        case _ =>
          JsError("Invalid JSON type")
      }

      /** @inheritdoc */
      override def writes(o: Image): JsValue = Json.toJson(o.sizes.toMap)
    }
}

final case class Size(path: String, height: Int, width: Int)

object Size {
  implicit val sizeFormat: Format[Size] = Json.format[Size]
}

Still works as before

scala> Json.parse("""
     | {
     | "large":{  
     |       "path":"http://url.jpg",
     |       "width":300,
     |       "height":200
     |     },
     |     "medium":{  
     |       "path":"http://url.jpg",
     |       "width":200,
     |       "height":133}}""")
res1: play.api.libs.json.JsValue = {"large":{"path":"http://url.jpg","width":300,"height":200},"medium":{"path":"http://url.jpg","width":200,"height":133}}

scala> res1.validate[Image]
res2: play.api.libs.json.JsResult[Image] = JsSuccess(Image(ListBuffer((large,Size(http://url.jpg,200,300)), (medium,Size(http://url.jpg,133,200)))),)

scala> Json.toJson(res1.validate[Image].get)
res3: play.api.libs.json.JsValue = {"large":{"path":"http://url.jpg","height":200,"width":300},"medium":{"path":"http://url.jpg","height":133,"width":200}}

But with the benefit that Size is now reflective of real Json, that is you can serialize and deserialize just Size values. Which makes it both easier to work with and think about.

TL;DR Commentary on reads in the First Example

Although I would argue that the first solution is somewhat inferior to the second solution, we did use some interesting idioms in the first implementation of reads that are very useful, in a more general sense, but often not well understood. So I wanted to take the time to go through them in more detail for those who are interested. If you already understand the idioms in use, or you just don't care, feel free to skip this discussion.

flatMap chaining

When we attempt to get the values we need out of valueMap, at any an all steps things can go wrong. We would like to handle these cases reasonably without catastrophic exceptions being thrown.

To accomplish this we use the Option value and common flatMap function to chain our computation. There are really two steps we do for each desired value, get the value out of valueMap and we force it to the proper type using the asOpt[T] function. Now the nice thing is that both valueMap.get(s: String) and jsValue.asOpt[T] both return Option values. This means that we can use flatMap to build our final result. flatMap has the nice property that if any of the steps in the flatMap chain fail, i.e. return None, then all other steps are not run and the final result is returned as None.

This idiom is part of general Monadic programming that is common to functional languages, especially Haskell and Scala. In Scala it is not often referred to as Monadic because when the concept was introduced in Haskell it was often explained poorly leading to many people disliking it, despite it in fact being very useful. Due to this, people are often afraid to use the "M word" with respect to Scala.

Functional Short Circuiting

The other idiom that is used in reads, in both versions, is short circuiting a function call by using the return keyword in scala.

As you probably know, use of the return keyword is often discouraged in Scala, as the final value of any function is automatically made into the return value for the function. There is however one very useful time to use the return keyword, that is when you are calling a function that represents a repeated call over something, such as the map function. If you hit some terminal condition on one of the inputs you can use the return keyword to stop the execution of the map call on the remaining elements. It is somewhat analogous to using break in a for loop in languages like Java.

In our case, we wanted to ensure certain things about the elements in the Json, like that it had the correct keys and types, and if at any point any of our assumptions were incorrect, we wanted to return the proper error information. Now we could just map over the fields in the Json, and then inspect the result after the map operation has completed, but consider if someone had sent us very large Json with thousands of keys that did not have the structure we wanted. We would have to apply our function to all of the values even if we knew we had an error after only the first application. Using return we can end the map application as soon we know about an error, without having to spend time apply the map application across the rest of the elements when the result is already known.

Anyway, I hope that little bit of pedantic explanation is helpful!

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Hey thanks a lot for the answer. It's very clean. Also thanks for the tips on validate and not using as, I hadn't thought of that before. I'm shutting off for the day but will comment some more tomorrow. Appreciate it
5

Suppose you want deserialize into the following case classes:

case class Size(name: String, path: String, width: Int, height: Int)
case class Image(sizes: List[Size])
case class Foo(..., image: Option[Image])

There are many ways to make this work via custom Reads implementations. I'm going to use the reads macro for Size:

implicit val sizeReads = Json.reads[Size]

Then, since the sizes aren't an actual array within the image object, I'll just make them into one to take advantage of the Reads[Size] I already have. I can transform a given JsValue that's being validated as an Image into a JsObject. I can then grab the fields from the JsObject which will be a Seq[(String, JsValue)]. In this case, the String is the size descriptor, and the JsValue is the object containing all the values for that size. I'll just merge them together into a single object, and make a JsArray out of the Seq.

From there, all I need to do is validate that JsArray as a List[Size], and map it to an Image.

implicit val imageReads = new Reads[Image] {
    def reads(js: JsValue): JsResult[Image] = {
        val fields: Seq[JsValue] = js.as[JsObject].fields.map { case (name, values) =>
            Json.obj("name" -> name) ++ values.as[JsObject]
        }

        JsArray(fields).validate[List[Size]].map(Image(_))
    }
}

Then Foo can also use the reads macro.

implicit val fooReads = Json.reads[Foo]

Example:

case class Foo(something: String, image: Option[Image])

val json = Json.parse("""{
    "something":"test",
    "image":{  
        "large":{  
            "path":"http://url.jpg",
            "width":300,
            "height":200
        },
        "medium":{  
            "path":"http://url.jpg",
            "width":200,
            "height":133
        }
    }
}""")

scala> json.validate[Foo]
res19: play.api.libs.json.JsResult[Foo] = JsSuccess(Foo(test,Some(Image(List(Size(large,http://url.jpg,300,200), Size(medium,http://url.jpg,200,133))))),)

Implementing a Writes[Image] is a little easier if you take advantage of Json.obj to mimic the structure of the output JSON you want. Since the output JSON doesn't actually use an array, we also need to merge the list of sizes back into a single object, which we can do using foldLeft.

implicit val writes = new Writes[Image] {
    def writes(img: Image): JsValue = {
        img.sizes.foldLeft(new JsObject(Nil)) { case (obj, size) =>
            obj ++ Json.obj(
                size.name -> Json.obj(
                    "path" -> size.path,
                    "width" -> size.width,
                    "height" -> size.height
                )
            )
        }
    }
}
Improve this answer

1 Comment

Hey that's a fantastic answer, it makes a lot of sense. I just compiled and found out I need a writer for this as well, because in class Foo I have (__ \ 'image).format[Option[Image] so I need to be able to write the object to JSON also. I edited my question to include my attempt but it's not working, any pointers on that also? Thanks again for the great response.
0

Maybe it is more normal with basic types. Only we define two classes:

final case class Size(path: String, width: Int, height: Int)
final case class Image(image: Map[String, Size])

implicit val sizeFormat: Format[Size] = Json.format[Size]
implicit val imageFormat: Format[Image] = Json.format[Image]

Then, run an example:

val json: JsValue = Json.parse("""
{
  "image":{  
    "large":{  
      "path":"http://url.jpg",
      "width":300,
      "height":200
    },
    "medium":{  
      "path":"http://url.jpg",
      "width":200,
      "height":133
    }
  }
}
""")

json.validate[Image]

You can get 

scala> json.validate[Image]
res13: play.api.libs.json.JsResult[Image] = JsSuccess(Image(Map(large -> Size(http://url.jpg,300,200), medium -> Size(http://url.jpg,200,133))),)

scala> json.validate[Image].get.image
res14: Map[String,Size] = Map(large -> Size(http://url.jpg,300,200), medium -> Size(http://url.jpg,200,133))

scala> json.validate[Image].get.image("large")
res15: Size = Size(http://url.jpg,300,200)

scala> json.validate[Image].get.image("large").path
res16: String = http://url.jpg

Also you could write:

scala> json.validate[Image].get
res18: Image = Image(Map(large -> Size(http://url.jpg,300,200), medium -> Size(http://url.jpg,200,133)))

scala> Json.toJson(json.validate[Image].get)
res19: play.api.libs.json.JsValue = {"image":{"large":{"path":"http://url.jpg","width":300,"height":200},"medium":{"path":"http://url.jpg","width":200,"height":133}}}
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.