Parallel combinators

ParT is a functional asynchronous non-blocking abstraction that allows you to build complex, data parallel coordination workflows using high-level constructs. This abstraction is controlled via parallel combinators, similar to the ones you can find in Orc, LINQ and Spark.

To start working with a ParT, first you need to import the `ParT module:

import ParT.ParT

Afterwards, you need to lift values into the ParT:

• liftv(x) lifts a value x into a ParT (liftv :: a -> Par[a])
• liftf(fut) lifts a future fut into a ParT (liftf :: Fut[a] -> Par[a])
• each(a) lifts an array a into a ParT (liftv :: [a] -> Par[a])

A ParT can be empty too, created by the function call empty[t]() :: Par[t]. A ParT is empty if it doesn't produce any meaningful result.

To combine ParTs, you use the ||| combinator (||| :: Par[a] -> Par[a] -> Par[a]), which returns a new ParT. ParTs are functional and asynchronous, which means that they are immutable and never stop the main thread of control.

Let's see an example with the concepts explained so far:

p1 = liftv(42)
pfut = liftf(a!foo())  -- asynchronous call returning a Fut[int]
p2 = each([1..10])
p3 = p1 ||| pfut ||| p2

Use the sequence combinator (>> :: Par[t] -> (t -> t') Par[t']) to create possibly parallel pipelines of work, similar to your pmap operation. There is also the bind combinator (bind :: Par[t] -> (t -> Par[t']) -> Par[t']) which is similar to bind combinator (>>=) in Haskell , except that this one can perform the work in parallel in the ParT.

Following the example above, let's increment each of the numbers in the ParT p3 with the >> combinator:

p3 >> fun (x: int) => x + 1

With the combinators explained so far you can create ParTs of ParTs, e.g. Par[Par[int]]. If you want to perform an operation on the elements contained in the ParT, you would have to create a closure that allows you to iterate over the elements, for instance:

p = liftv(liftv(42))  -- Par[Par[int]]
p >> fun (x: Par[t])
     x >> fun (y: int) => y + 1
   end

Instead of doing this, the join combinator (join :: Par[Par[t]] -> Par[t]) flattens the ParT. The following code performs the same thing as the example above:

p = liftv(liftv(42))  -- Par[Par[int]]
join(p) >> fun (x: int) => x + 1

To asynchronously filter elements from a ParT use the filter combinator (filter :: (a -> bool) -> Par[a] -> Par[a]). If none of the elements satisfy the condition, the result is an empty ParT. Otherwise, the result is a ParT that will eventually contain the result of the elements that satisfy the condition.

If you want to extract the results from the parallel abstraction, you can use the extract combinator (extract :: Par[t] -> [t]) which returns the values of the ParT into an array. This combinator is the only one that blocks the current actor until all the items in the ParT have been realised.

There are more combinators in the ParT module that allows you to use your common relational combinators, such as intersection, union, distinct, groupJoin, groupByKey, aggregate, all, any, max, min, etc.