Scalar Conference is the first free to attend scala conference. At least first in central Europe I know.

I had a pleasure to be a part of it as an attendee and a listener. I must say it was well organized. The organizers –  Softwaremill – top notch software house from Warsaw – proved that are real professionals in this matter. There was up to 300 attendees. The atmosphere was amazing because of many reasons: subjects were great, I met many friends and the most important I am just in the middle of a transfer from Allegro to Softwaremill :) You can imagine that I was very excited by meeting my new colleagues!

This was the one of those conferences where polish speakers were much more better than abroad guests.

  • Tomek Nurkiewicz is fully professional stage beast. His english is just as he was born in Birmingham Palace. He was talking about dark sides of scala, some complex notations and implementations of standard library.
  • Adam Warski is second great presenter, experienced in giving lectures in english. Adam was talking about spray.io – it looked to me as micro-framework in scala, I’ve already know a few in other languages – ratpack in groovy, flask, bottle in python, express in nodejs – all are very super tools!
  • Then comes Konrad Malawski I know he is Pole but listening to him you are sure he comes from Chelsea – a district of Kraków or maybe London. Konrad pretends to me as a hardworking guy. He became a Akka developing team member. He was talking about persistence in akka world – you might want to persists Actor’s mailboxes.
  • Grzegorz Kossakowski was a Typesafe star on scalar.  On daily basis he is scala compiler developer. He proved why java8 lambas being implemented on top of invokedynamic are not faster then those from scala being implemented on classsical java approach – anonymous class implementing interface with one method.
  • Very interesting guys from ICM presting hadoop + scala, scoobi and scalding frameworks for hadoop. What inspired me the most was the scala REPL for hadoop.
  • And there was Łukasz Kuczera with presentation on scalaz – functional library to scala.

All above were very interesting. Guys were prepared and I was focused all time and understood everything.

  • Grzegorz Kubiak had a great topic but I think he didn’t practise before exhibition.
  • Stefan Zegier was correct talking about slick – it look for me as a orm in scala.
  • Jon Pretty crazy guy was funny but might be misunderstood by most with his algebra proofs :)
  • Mathias Nehlsen didn’t treat poles seriously or maybe  he did just boring presentation. And the two worst presentation where underscore magic – underscore in scala? come on! we’ve got scala compiler developer on place! and macro based type providers in scala… no code, no examples – no reward. I still don’t know what are type providers — totally a waste of time.

After that amazing conference I went to after party with friends from Allegro, Polidea and Javeo where I met all other colleges from Softwaremill. That was a great time. I hope that Pedrowaty a chef of solution architects in Allegro (he was present on conference) changed his mind about scala – it’s not a dead end any more. This is a great article about Transitioning to scala.

The conference was free but with this high quality I would really not bother to pay for it.

Posted

I had an amazing opportunity to attend training conducted by Paweł Kozłowski author of Mastering Web Application Development with AngularJS.

Training was two days long only … only because Paweł was excellently prepared and it was pleasure to listen to him. I met grunt, karma, jasmine and of course angular. With my growing knowledge of js I am starting to like this flexibility it gives. Here you can find our log of source code and Paweł’s presentation made in revealJs!

We maintain our front end in backbone js after a chat to my college I discovered that we are facing memory leaks with out webapp. And that is an issue to handle in short time. Maybe we will use marionette js or just move completely to angular :) (lots of work ahead, lots of reports to rewrite)

Codeschool.com is a very pleasant place to learn new things. I completed there “chrome dev tools training” to learn how to profile and dump heap in js – it might be help fully. This was the way I saw how our app leaks…

Posted

Hi, I’ve dropped by just to invite you to Git Kata 2 (git workshop in kata format) meeting in Warsaw, Poland on Dec 8, 2013.

I will try to convince people to use legit and github. I will describe shortly my participation in fabtools – opensource project.

All details may be found here

 

Here I am

Posted

Usage example of Cascading Tube

EXAMPLE I
Step 0 is pure cascading flow example Cascading.

Step 0.

Short description of method: it takes logs from one source and produces tree different reports

public FlowDef flow(Tap pv, Tap trans, Tap transToFirstSource, Tap transToLastSource, Tap transDistributedAmongSources) {
// agregation
Pipe transPipe = new Each("trans", new Fields("t_quantity", "t_itemCost"), new ExpressionFunction(new Fields("t_income"),
"(double) t_quantity * (double) t_itemCost", Double.class), Fields.ALL);
transPipe = new AggregateBy(transPipe, new Fields("t_userId", "t_transactionId"), new FirstBy("t_createTime"),
new SumBy(new Fields("t_quantity"), new Fields("t_quantity"), Integer.class),
new SumBy(new Fields("t_income"), new Fields("t_income"), Double.class));

// divide
Pipe pvPipe = new Pipe("pv");
pvPipe = new GroupBy(pvPipe, new Fields("p_userId"), new Fields("p_createTime"));
Pipe visitPipe = new Every(pvPipe, new GetVisit(selfReferred()), Fields.RESULTS);

// merge
Pipe crossTransWithVisits = new CoGroup(visitPipe, new Fields("p_userId"), transPipe, new Fields("t_userId"));
crossTransWithVisits = new Each(crossTransWithVisits, new Fields("start", "t_createTime"), new OlderThan());
crossTransWithVisits = new Checkpoint(crossTransWithVisits);

return FlowDef.flowDenew Fields().setName("ConversionAttribution").addSource(pvPipe, pv).addSource(transPipe, trans)
.addTailSink(assignTransactionToFirstVisit(crossTransWithVisits), transToFirstSource)
.addTailSink(assignTransactionToVisit(crossTransWithVisits), transToLastSource)
.addTailSink(distributeTransactionAmongVisits(crossTransWithVisits), transDistributedAmongSources);
}

|
|
|
|
\/

Step 1a.

You can shorten your code by adding helper class with and use “f(_)” instead of “new Fields(_)”:

static public Fields f(String... names) {
return new Fields(names);
}

static public SumBy sum(String field, Class<?> resultType) {
return new SumBy(f(field), f(field), resultType);
}

static public SumBy sum(String field) {
return sum(field, Double.class);
}

static public SumBy sumInt(String field) {
return sum(field, Integer.class);
}

static public CountBy count(String outField) {
return new CountBy(f(outField));
}

replacing

new Fields(_) == > f(_)
new FirstBy(field) == > first(field)
new SumBy(new Fields("t_quantity"), new Fields("t_quantity"), Integer.class) == > sumInt("t_quantity")
new SumBy(new Fields("t_income"), new Fields("t_income"), Double.class) == > sum("t_income")

—– OR
|
|
|
\/

Step 1b.

Or you can try to use Cascading Tube wrapper, that implements builder pattern upon cascading and makes your code shorter, concise, readable and intuitive?

(This method does exactly same thing as that above – it’s only rewritten to “Cascading Tube”)
Add some imports first

import jj.tube.Aggregators._
import jj.tube.Io._
import jj.tube.Tube._
import jj.tube._

and put this method into your class

def flow(pv: Io, trans: Io, transToFirstSource: Io, transToLastSource: Io, transDistributedAmongSources: Io, transTrap: Io, pvTrap: Io) = {
val transPipe = Tube("trans")
.multiply("t_quantity", "t_itemCost", "t_income")
.aggregateBy(("t_userId", "t_transactionId"), first("t_createTime"), sumInt("t_quantity"), sum("t_income"))

val pvPipe = Tube("pv")
.groupBy("p_userId", "p_createTime")
.every(buffer = new GetVisit(selfReferred))

val crossTransWithVisits = Tube("crossVisitAndTrans", pvPipe).coGroup("p_userId", transPipe, "t_userId", new RightJoin)
.discard("p_userId")
.groupBy(("t_userId", "t_transactionId"), "start")
.every(buffer = new FillMisingMatchingVisit)
.filter(("start", "t_createTime"), new OlderThan)
.checkpoint

FlowDef.flowDef().setName("ConversionAttribution").addSource(pvPipe, pv)
.addSource(transPipe, trans)
.addTailSink(assignTransactionToVisit(crossTransWithVisits, FIRST), transToFirstSource)
.addTailSink(assignTransactionToVisit(crossTransWithVisits, MATCHING), transToLastSource)
.addTailSink(distributeTransactionAmongVisits(crossTransWithVisits), transDistributedAmongSources)
.addTrap("trans", transTrap)
.addTrap("pv", pvTrap)
}

Which one do you like to read more?

Posted

And so many ideas. I have found an interesting reading nathanmarz.com. I was just seeking some hadoop related solution and found that guy. Looks interesting, so I will follow his twitts for a while.

  • I use feedly instead of google reader, I hate it on my mobile when it comes to run it first time after upgrade, it always asks me to login to google reader, I don’t know how this will end
  • I am almost ready with scala course from coursera, even though I don’t feel comfortable with scala. I tried to modify playframework scala todolist and failed with that “syntactic sugar”. [my scala play todolist]
  • As we are adopting Cascading into our project (We believe it will replace apache pig for our mapreduce computations) I finished “learn.cascading” — (link in cascading documentation page) there are given test in junit and you are asked to write an implementation for it — exactly as in groovy-koans. [Cascading isn’t as intuitive as pig but running it’s “ultrafast” tests -comparing to pigunit- makes it worth spending time on it. We use cloudera distro of hadoop and in latest CDH4.2.1 there is still pig in version 0.10, maybe pig current apache stable version 0.11 tests run faster?]
  • This week I’ve started to use idea ide, I was a big fun of netbeans working in Orange, I am fun of sts eclipse working in allegro (except the moments its window simply disappears in my mint nadia 14 – I hope it was missing repo or plugin issue) but the eclipse groovy plugin does not highlight unused imports… and idea does.
  • I wish I know how to “keep fire under motivations”. I wish to run something. I wish to participate in something cool. Of course my work is something that kind but still I feel there is a room for more. I am 30 right now and the most important thing I’ve changed in my point of view is about people. It is very important to have great guys around. The only way to build something working fine is to do  it with a team. Take a look on stendi.pl. I thought I am able to build such a shop myself. But I was wrong. I see a great team working on it. I see testers hard work. And I know now how big cost is to build something shiny.

Posted