Structured data

Basic vector operations

Vectors are immutable: once you have a vector, you can not change it. You can, however, easily create new vectors based on a vector:

(conj [1 2 3] 4)          ;=> [1 2 3 4]
(assoc [1 2 3 4] 2 "foo") ;=> [1 2 "foo" 4]

conj adds a value to a collection. Its behaviour depends on the type of collection: with vectors, it adds the value to the end of the vector. To be exact, conj does not change the given vector. Instead, it returns a new vector, based on the given vector, with the new element appended to the end.

(cutify []) => ["<3"]
(cutify [1 2 3]) => [1 2 3 "<3"]
(cutify ["a" "b"]) => ["a" "b" "<3"]

Vectors: A Postmodern Deconstruction

Another way of extracting values from a vector is by destructuring it:

(let [[x y z] [1 2 3 4 5 6]]
  (str x y z))
;=> "123"

Here, instead of giving a name to the vector [1 2 3 4 5 6], we indicate with the brackets in [x y z] that we want to destructure the vector instead. Inside the brackets, we give names to the first three elements of the vector. x will be given the value of the first element, 1; b will be 2 and c will be 3. The concatenation of these values that str returns is "123".

You can destructure function parameters directly. For an example, take the following function:

(defn sum-pairs [first-pair second-pair]
  [(+ (first  first-pair) (first  second-pair))
   (+ (second first-pair) (second second-pair))])

The function takes two vectors and sums the elements pairwise:

(sum-pairs [42 5]   [-42 -5])   ;=> [0 0]
(sum-pairs [64 256] [-51 -219]) ;=> [13 37]

sum-pair is not very pretty to look at. We can spiff it up by taking out the elements of its parameter vectors by destructuring them:

(defn sum-pairs [[x1 y1] [x2 y2]]
  [(+ x1 x2) (+ y1 y2)])

sum-pairs still takes two parameter vectors, but now it does not give names to its parameters. Instead, it gives names to their first two elements by destructuring the parameters. We could have also destructured the parameters with a let.

We are a legion

count can be used to find out the amount of elements in a collection.

(count [1 2 3]) ;=> 3
(count {:name "China Miéville", :birth-year 1972}) => 2
(count ":)") => 2

As we can see, count tells the amount of keys for a map and the amount of elements for a vector. It can also be used to find out the length of a string.

Let’s define some authors and a couple of books with maps and vectors.

(def china {:name "China Miéville", :birth-year 1972})
(def octavia {:name "Octavia E. Butler"
              :birth-year 1947
              :death-year 2006})
(def friedman {:name "Daniel Friedman" :birth-year 1944})
(def felleisen {:name "Matthias Felleisen"})

(def cities {:title "The City and the City" :authors [china]})
(def wild-seed {:title "Wild Seed", :authors [octavia]})
(def embassytown {:title "Embassytown", :authors [china]})
(def little-schemer {:title "The Little Schemer"
                     :authors [friedman, felleisen]})

(title-length cities)         ;=> 21
(title-length wild-seed)      ;=> 9
(title-length little-schemer) ;=> 18

(author-count cities)         ;=> 1
(author-count wild-seed)      ;=> 1
(author-count little-schemer) ;=> 2

(multiple-authors? cities)         ;=> false
(multiple-authors? wild-seed)      ;=> false
(multiple-authors? little-schemer) ;=> true

Adding Values to a Map

(assoc a-map a-key a-value) sets the value of a-key in a-map to be a-value.

(assoc {:a 1} :b 2) ;=> {:b 2, :a 1}
(assoc {:a 1} :a 2) ;=> {:a 2}

Let’s add some information to a book:

(assoc cities :awards ["Hugo", "World Fantasy Award",
                       "Arthur C. Clarke Award",
                       "British Science Fiction Award"])
;=> {:awards ["Hugo" "World Fantasy Award" "Arthur C. Clarke Award"
;             "British Science Fiction Award"]
;    :title "The City and the City"
;    :authors [{:birth-year 1972, :name "China Miéville"}]}

Vectors are an associative data structure, so assoc also works with them.

;index: 0 1 2                0   1   2
(assoc [3 2 1] 1 "~o~") ;=> [3 "~o~" 1]

Here the key that you give as a parameter is the index that you want to change.

Assoc does not actually change the original data structure, but instead returns an updated version of it.

(let [original [1 2 3 4]
      new      (assoc original 2 "foo")]
  original)
;=> [1 2 3 4]

(add-author little-schemer {:name "Gerald J. Sussman"})
;=> {:title "The Little Schemer"
;    :authors [{:birth-year 1944, :name "Daniel Friedman"}
;              {:name "Matthias Felleisen"}
;              {:name "Gerald J. Sussman"}]}
(add-author {:authors [{:name "Juhana"}]} {:name "Jani"})
;=> {:authors [{:name "Juhana"} {:name "Jani"}]}

The keys and values of a map can be of any data type, and one map can contain any number of different data types as both keys and values.

(contains? a-map a-key) can be used to check if a-map has a value for a-key.

(contains? {"a" 1} "a")   ;=> true
(contains? {"a" 1} 1)     ;=> false
(contains? {"a" nil} "a") ;=> true
(contains? cities :title) ;=> true
(contains? cities :name)  ;=> false

(alive? china)   ;=> true
(alive? octavia) ;=> false

Sequences

Before talking about map, we need to introduce a new concept: the sequence. Many of Clojure’s functions that operate on vectors and other collections actually operate on sequences. The (seq collection) function returns a sequence constructed from a collection, such as a vector or a map.

Sequences have the following operations:

• (first sequence) returns the first element of the sequence.

• (rest sequence) returns the sequence without its first element.

• (cons item sequence) returns a new sequence where item is the first element and sequence is the rest.

(seq [1 2 3])          ;=> (1 2 3)
(seq {:a 42 :b "foo" :c ["ur" "dad"]})
                       ;=> ([:a 42] [:c ["ur" "dad"]] [:b "foo"])
(first (seq [1 2 3]))  ;=> 1
(rest (seq [1 2 3))    ;=> (2 3)
(cons 0 (seq [1 2 3])) ;=> (0 1 2 3)

Here you can see the printed form of sequences, the elements inside ( and ). This has the consequence that copying (1 2 3) back to the REPL tries to call 1 as a function. The result is that you can not use the printed form of a sequence as a value like you could with vectors and maps.

Actually, the sequence functions call seq on their collection parameters themselves, so we can just write the above examples like this:

(first [1 2 3])  ;=> 1
(rest [1 2 3])   ;=> (2 3)
(cons 0 [1 2 3]) ;=> (0 1 2 3)

The map function

(map function collection) takes two parameters, a function and a sequenceable collection. It calls the function on each element of the sequence and returns a sequence of the return values.

(defn munge [x]
  (+ x 42))

(map munge [1 2 3 4])
;=> ((munge 1) (munge 2) (munge 3) (munge 4)) ; [note below]
;=> ( 43        44        45        46)

Note: You can’t paste the result line (or the middle one) to the REPL, as it is the printed form of a sequence.

(element-lengths ["foo" "bar" "" "quux"])  ;=> (3 3 0 4)
(element-lengths ["x" [:a :b :c] {:y 42}]) ;=> (1 3 1)

Earlier, we briefly introduces the fn special form that can be used to create functions. This is useful when you want a function that is only visible in the definition of another function. Quite often you want to use let to give name to this helper function.

Let’s rewrite the example above in this style:

(defn mungefy [a-seq]
  (let [munge (fn [x] (+ x 42))]
    (map munge a-seq)))

Now the function munge is only visible inside the definition of mungefy. It should work like the previous one.

(mungefy [1 2 3 4]) ;=> (43 44 45 46)

(second-elements [[1 2] [2 3] [3 4]]) ;=> (2 3 4)
(second-elements [[1 2 3 4] [1] ["a" "s" "d" "f"]])
;=> (2 nil "s")

When you have a sequence of maps, the fact that :keywords are also functions can be helpful.

(:name {:name "MEEEE", :secret "Awesome"}) ;=>  "MEEEE"

You can therefore use a :keyword as the function parameter of map.

(let [people [{:name "Juhana", :age 3}
              {:name "Ilmari", :age 42}
              {:name "Jani", :age 72}
              {:name "King of All Cosmos" :age -6}]]
  (map :age people))
;=> (3 42 72 -6)

(def china {:name "China Miéville", :birth-year 1972})
(def octavia {:name "Octavia E. Butler"
              :birth-year 1947
              :death-year 2006})
(def friedman {:name "Daniel Friedman" :birth-year 1944})
(def felleisen {:name "Matthias Felleisen"})

(def cities {:title "The City and the City" :authors [china]})
(def wild-seed {:title "Wild Seed", :authors [octavia]})
(def embassytown {:title "Embassytown", :authors [china]})
(def little-schemer {:title "The Little Schemer"
                     :authors [friedman, felleisen]})

(def books [cities, wild-seed, embassytown, little-schemer])

(titles [cities]) ;=> ("The City and the City" )
(titles books)
;=> ("The City and the City" "Wild Seed"
;    "Embassytown" "The Little Schemer")

Okey, so now that map has been gone over, let’s see the definition of all-author-names again.

(defn author-names [book]
  (map :name (:authors book)))

(defn all-author-names [books]
  (set (apply concat (map author-name books))))

author-names returns the names of the authors of a single book.

(author-names cities)         ;=> ("China Miéville")
(author-names little-schemer) ;=> ("Daniel Friedman" "Matthias Felleisen")

Since this is just a helper function used inside all-author-names we can move it inside by using let and fn.

(defn all-author-names [books]
  (let [author-names
         (fn [book] (map :name (:authors book)))]
    (set (apply concat (map author-names books)))))

The definition of all-author-names still has some mysterious words like set, apply and concat in it. Let’s see what would happen without them.

(map author-names [cities]) ;=> (("China Miéville"))
(map author-names [cities, wild-seed]) ;=> (("China Miéville") ("Octavia E. Butler"))

So first of all we would get every books authors inside a sequence. To fix this, we need to concatenate the sequences. To do this, there is concat.

(concat ["China Miéville"] ["Octavia E. Butler"]) ;=> ("China Miéville" "Octavia E. Butler")

This looks like what we want. However, if we simply try to use (concat (map author-names books)), we get the following problem:

(concat (map author-names [cities, wild-seed]))
;=> (concat (("China Miéville") ("Octavia E. Butler")))
;=> (("China Miéville") ("Octavia E. Butler"))

So we end up only giving concat one argument and it simply returns the argument. What we want is to give the elements of (map author-names books) to concat as arguments.

No worries, there is a way to do this. Let’s check out apply.

Apply Now, Redux

(apply function a-seq) applies function to the arguments in a-seq. Here’s an example:

(apply + [1 2 3])
;=> (+ 1 2 3)
;=> 6

And here’s another with concat:

(apply concat [["China Miéville"] ["Octavia E. Butler"]])
;=> (concat ["China Miéville"] ["Octavia E. Butler"])
;=> ("China Miéville" "Octavia E. Butler")

More generally, apply works like this:

(apply function [arg1 arg2 arg3 ...]) => (function arg1 arg2 arg3 ...)

(repeat 5 "*") ;=> ("*" "*" "*" "*" "*")
(repeat 3 "~o~") ;=> ("~o~" "~o~" "~o~")

(stars 1) ;=> "*"
(stars 7) ;=> "*******"
(stars 3) ;=> "***"

(monotonic? [1 2 3])     ;=> true
(monotonic? [0 1 10 11]) ;=> true
(monotonic? [3 2 0 -3])  ;=> true
(monotonic? [3 2 2])     ;=> true    Not strictly monotonic
(monotonic? [1 2 1 0])   ;=> false

So now we can put all authors into a single list. There’s just one problem left. What is that? Well, let’s see what happens if we put together everything seen so far.

(apply concat (map author-names books))
;=> ("China Miéville" "Octavia E. Butler"
;    "China Miéville" "Daniel Friedman"
;    "Matthias Felleisen")

We had two books by China Miéville, so his name is in the resulting sequence twice. But when we want to see the authors, we are usually not interested in duplicates. So lets turn the sequence into a data structure that supports this.

Set

Our last major data structure is the set. It is an unordered collection of items without duplicates.

(set ["^^" "^^" "^__*__^"]) ;=> #{"^__*__^" "^^"}
(set [1 2 3 1 1 1 3 3 2 1]) ;=> #{1 2 3}

The textual form of a set is #{an-elem another-elem ...} and you can convert another collection into a set with the function set.

Sets have three basic operations:

You can check whether a set contains an element with the function contains?:

(def games #{"Portal", "Planescape: Torment",
             "Machinarium", "Alpha Protocol"})

(contains? games "Portal") ;=> true
(contains? games "RAGE")   ;=> false
(contains? games 42)       ;=> false

(conj set elem) adds elem to set if it does not already have elem:

(conj #{:a :b :c} :EEEEE) ;=> #{:a :c :b :EEEEE}

Nothing happens if elem is already a member of set:

(conj #{:a :b :c} :a)     ;=> #{:a :c :b}

You can also add multiple elements by giving conj additional arguments:

(conj #{:a :b :c} :d :e)  ;=> #{:a :c :b :d :e}

Finally, (disj set elem) removes elem from set if it contains elem:

(disj #{:a :b :c} :c) ;=> #{:a :b}
(disj #{:a :b :c} :EEEEE) ;=> #{:a :c :b}
(disj #{:a :b :c} :c :a) ;=> #{:b}

(toggle #{:a :b :c} :d) ;=> #{:a :c :b :d}
(toggle #{:a :b :c} :a) ;=> #{:c :b}

If you want to know the size of a set, count also works with sets.

(count #{1 2 3}) ;=> 3
(count (set [1 2])) ;=> 2

(contains-duplicates? [1 1 2 3 -40]) ;=> true
(contains-duplicates? [1 2 3 -40]) ;=> false
(contains-duplicates? [1 2 3 "a" "a"]) ;=> true

Our books looked like this:

(def friedman {:name "Daniel Friedman" :birth-year 1944})
(def felleisen {:name "Matthias Felleisen"})

(def little-schemer {:title "The Little Schemer"
                     :authors [friedman, felleisen]})

Now we can understand the whole implementation of all-author-names. We use

• fn to introduce a helper function,
• keywords to index the books,
• map to get all authors from a single book
• let to give a name to our helper function,
• map to apply the helper function to all the given books, and
• construct a set with the set function to get rid of duplicates.

(defn all-author-names [books]
  (let [author-names
         (fn [book] (map :name (:authors book)))]
    (set (apply concat (map author-names books)))))

Calling our function returns the desired set:

(all-author-names books)
;=> #{"Matthias Felleisen" "China Miéville"
;     "Octavia E. Butler" "Daniel Friedman"}

String Representation for Books

Now that we have defined these books, I would like to have a readable string representation for them. Let’s start by defining a representation for a single author.

(author->string felleisen) ;=> "Matthias Felleisen"
(author->string friedman)  ;=> "Daniel Friedman (1944 - )"
(author->string octavia)   ;=> "Octavia E. Butler (1947 - 2006)"

Now we have a string representation for a single author. Some of our books had multiple authors, so we need to figure out a way to give a string representation for multiple authors. To do this, we need a handy helper function.

Sometimes you want to add something in between the elements of a sequence. For that, there is (interpose separator a-seq), which returns a new sequence that has separator between each element of a-seq.

(interpose ":" [1 2 3])        ;=> (1 ":" 2 ":" 3)
(interpose " and " ["a", "b"]) ;=> ("a" " and " "b")
(interpose ", " [])            ;=> ()

(apply str (interpose " and " ["a", "b"])) ;=> "a and b"

With this, it shouldn’t be too hard to get a nice representation for a sequence of authors.

(authors->string (:authors little-schemer))
;=> "Daniel Friedman (1944 - ), Matthias Felleisen"
(authors->string #{octavia})          ;=> "Octavia E. Butler (1947 - 2006)"
(authors->string #{})                 ;=> ""
(authors->string #{octavia, friedman})
;=> "Octavia E. Butler (1947 - 2006), Daniel Friedman (1944 - )"
;   order doesn't matter

Now that we can handle the case of multiple authors, we can move on to the string representation of a single book.

(book->string wild-seed) ;=> "Wild Seed, written by Octavia E. Butler"
(book->string little-schemer)
;=> "The Little Schemer, written by Daniel Friedman (1944 - ), Matthias Felleisen"
;                                   ^-- order doesn't matter

And finally, we can define a string representation for a sequence of books.

(books->string []) ;=> "No books."
(books->string [cities])
;=> "1 book. The City and the City, written by China Miéville (1972 - )."
(books->string [little-schemer, cities, wild-seed])
;=> "3 books. The Little Schemer, written by Daniel Friedman (1944 - ), Matthias Felleisen. The City and the City, written by China Miéville (1972 - ). Wild Seed, written by Octavia E. Butler (1947 - 2006)."

Keeping your vectors

map and filter always return sequences, regardless of the collection type given as a parameter. Sometimes, however, you want the result to be a vector. For an example, you may want to index the vector afterwards. In this situation, you can use mapv and filterv, which are variants of map and filter that always return vectors.

(mapv ... [...])    ;=> [...]
(filterv pos? [-4 6 -2 7 -8 3])  ;=> [6 7 3]
(filterv pos? #{-4 6 -2 7 -8 3}) ;=> [3 6 7]
(mapv ... #{...})   ;=> [...]