Dotty – Union Data Types

Hello Folks, As we know that dotty is new Scala compiler (also know as Scala 3.0). Which is coming with some new features and improvements. To get more details about the dotty and its environment setup. Please follow our beginner guide blog.

In this blog, I will describe you about newly introduced Union data type of dotty and its  various properties.

A union is data type, which is denoted by “|” pipe infix operator and it looks like A | B. A union type (A | B) has a values of all values of type A and also all values of type B.

The primary reason for introducing union types in Scala is that they allow us to guarantee that for every set of types, we can always form a finite least upper bound (lub). This is both useful in practice as well as in theory. The type system of Scala 3 (i.e. dotty) is based on the DOT calculus, which has also a union data types.

A union type can also be join to a non union types and form a finite join types. Like we can define join of a union type T1 | … | Tn as the smallest intersection type of base class instances of T1,…,Tn.

Syntactically, unions follow the same rules as intersections, but have a lower precedence than intersection(&).

Example of Dotty’s Union Types –

You can find the complete example of Dotty Union Type from here.

In this example, we have two case class models:

  • UserName
  • Password

Both have a common method lookup(). Which display the UserName and Password info on basis of its type calling.

We have also defined a getLoginInfo() method which takes a union type as an argument and return a union type value.

def getLoginInfo(loginInfo: UserName | Password): Password | UserName = {
    loginInfo match {
      case u@UserName(_) =>
        u.lookup
        u
      case p@Password(_) =>
        p.lookup
        p
    }
  }

Union types holds the commutative properties example: A | B =:= B | A

  val value1: UserName | Password = getLoginInfo(UserName("Narayan"))
  val value2: Password | UserName = getLoginInfo(UserName("Narayan"))
  if(value1 == value2) println("Commutative properties satisfy") else 
            println("Commutative properties does not satisfy")

Other Important points are –

  • Union type also supports distributive properties over the intersection (&) type.
  • Join operation on union types guarantees that the join is always finite.
  • The members of a union type are also the members of its join.
  • Union type “|” operator has lower precedence than both intersection(&) and type inference pattern (“:”) operators.
  • We can effectively use union type in pattern matching.

Stay Tunes, happy learning  !!!

References:

https://dotty.epfl.ch/docs/reference/new-types/union-types.html

 

Knoldus-blog-footer-image

The Role of Securities Market Place in trading

Hello folks, in the previous blog, I had discuss about some essential trading concepts and terminology. In this blog, I am going to describe you about the structure and components of the Securities market place.

In any street market, the goods that are sold are manufactured outside of the marketplace and the characters involved are those who visit the marketplace in order to buy goods and those who reside within the marketplace in order to sell those goods.

Similarly, we can do a direct comparison of street market with the securities marketplace as follows:

  • The goods (securities) are originated by organisations (issuers of securities) outside the marketplace.
  • Investors visit the marketplace in order to buy and sell securities, either directly or via an agent.
  • The STOs reside within the marketplace in order to sell securities to and buy securities from investors and agents.

So in layman term, we can say that the securities marketplace is place which provides the infrastructure to allow trading to occur, within a regulated environment.

Before going into more detail, first we need to understand the term Securities. Like what is securities, how it comes into securities marketplace, what are the types etc.

Securities: Organisations occasionally need to raise cash (or capital) in order to expand their business through, for example, buying new premises, building new factories or acquiring other companies. The options open to such organisations for raising the necessary capital includes:

  1. Borrowing cash from banks.
  2. Selling a part of their existing business.
  3. Selling part ownership in the company (issuing shares).
  4. Borrowing cash from investors (issuing bonds).

So in this both shares and bonds are generally known as securities.

The role of the securities marketplace here are:

  • Facilitates the process of bringing new securities to the marketplace.
  • Provides a structured and regulated method of buying and selling existing securities for the protection of the investors.

Common Securities Types: There are two most common types of securities issued by companies are equity(also known as shares or stock) and debt(also known as bonds).

Equity: An equity represents ownership in a company, the purchase of shares by an investor gives the investor partial ownership in that company (a ‘share’ in the company). The issuer effectively dilutes the ownership of the company by spreading ownership across hundreds or thousands of investors, dependent upon the size of the company and the number of shares in issue.

The issue of equity provides the company with ‘permanent’ cash, in other words when the company sells shares initially there is no intention to repay to shareholders the capital received (however, on an exceptional basis an issuer may decide to buy back shares from the shareholders).

If an investor wishes to sell his shareholding, the shares are not normally purchased by the issuer but are sold in the marketplace and ultimately purchased by another investor.

Debt: A debt issue reflects a borrowing of a specific amount of cash by an issuer (such as a company, government or government agency) in a specified currency. The purchase of bonds by an investor is effectively on a promise by the issuer to repay the capital to the investor at a future point in time(i.e. maturity date). Also issuer will make a periodic payments of interest to the investor at predefined points in time and (normally) at a predefined rate of interest. The interest on bonds is commonly referred to as coupon.

If an investor wishes to sell his bond-holding, the bonds are not normally purchased by the issuer but are sold in the marketplace to another investor, as is the case for equities.

In next blog, I will describe some other components of the securities marketplace. So stay tune.

Happy Learning !!!

References:

Book: Securities Operations: A Guide to Trade and Position Management by Michael Simmons

 
Knoldus-blog-footer-image

The essential trading concepts and Terminology

Hello folks, welcome to my trading concepts blog series. In this blog, I will describe and introduce some trading related terminology and its importance in trade life cycle.

For any company in any industry, remaining in control of its goods and cash is fundamental to successful and efficient operation of its business. This involves maintaining up-to-date internal records of trading activity, deliveries and receipts of goods, as well as payments and receipts of cash, thereby enabling the prediction of future stock and cash flows and the reconciliation of internal records with external entities, including goods held in warehouses and cash held at banks. I am going to share with you a list of different terminology used within the securities industry for trade agreement purpose.

Many of these phrases and terms used within the securities industry can be related to those used in the outside world. But it is important to understand the meaning of the components of a trade in order to understand their impact on trade agreement and its life cycle.

These are following terminology widely used in the securities industry:

  • Trade date: The date the parties to the trade agreed to trade. It is also known as date of trade execution.
  • Operation: The type and direction of the trade, e.g. buy or sell, lend or borrow.
  • Quantity: The number of units of the goods being exchanged.
  • Goods: The specific goods being exchanged. This is in which STOs (Securities trading organisations) invests. It is also referred to as securities or instruments.
  • Price: The price of each unit being exchanged.
  • Supplier: The entity with whom the trade has been executed (the deliverer of goods and the receiver of cash). The party with whom the trade is conducted is also known as the counter party, whether buying or selling, lending or borrowing.
  • Delivery date: The agreed intended date of delivery of goods by the supplier and payment of cash by the buyer. It is also known as value date or contractual settlement date.
  • Cash due: The cash value of the trade due to be paid to the supplier upon delivery of the goods.
  • Required location of goods: From the buyer’s perspective, the desired storage location of the goods.
  • Cash to be paid from: From the buyer’s perspective, the specific bank from which payment is to be made.

Apart from these, we have another few important terms of trading that we should understand to know that, how the trading operations works under the hood.

Role of custodian in trading: To describe the custodian term. Lets take an example. An STO based in Stockholm may choose to invest in a Japanese security. There may be nothing to prevent the Stockholm-based STO taking delivery of the securities in Stockholm, however, from an efficiency and cost perspective STOs typically require delivery to occur in the normal place of issue and delivery of the securities, in this example Tokyo. The alternative is that securities would be delivered from Tokyo to Stockholm, with lengthy delivery time-frames and the costs (e.g. insurance) of such a delivery to be considered, and with the seller likely to demand payment for the securities before the securities leave the seller’s possession. If the buyer agrees to this arrangement, he would be taking a risk of being without both securities and cash an unacceptable risk to most STOs.

In order to minimize this risk and to exchange securities and cash in the most efficient manner for all securities in which they invest, STOs typically utilize local agents to exchange securities and cash on their behalf. These local agents are often referred to as custodians. Securities are held within a custodian securities account (also known as a depot account) and cash is held within a custodian cash account (also known as a nostro account).

Settlement of Trade: The exchange of securities and cash is known as settlement within the securities industry. The act of settlement should be viewed as no different from buying or selling goods in one’s personal life. For instance, most people are hesitate to pay the purchase cost of a car without taking delivery of the car at the same time. Conversely, most people would not feel happy about handing over a car that they were selling without receiving the buyer’s cash at the same time. The risk, applicable to both buyer and seller, is that at one point in time they may be in possession of neither asset, whether goods or cash.

STOs are typically very risk averse and they try to avoid these situations whenever possible. The most efficient and risk-free method of settlement is known as Delivery versus Payment (DvP), whereby simultaneous exchange of securities and cash is effected between buyer and seller (through their custodians), the seller not being required to deliver securities until the buyer pays the cash and the buyer not being required to pay cash until the seller delivers the securities, thereby ensuring that both parties are protected.

In next blog, I will explain you about the securities market place. So stay tune.

Happy Learning !!!

References:

Book: Securities Operations: A Guide to Trade and Position Management by Michael Simmons

 


Knoldus-blog-footer-image

Integration of a DocuSign service in Scala.

DocuSign is the leading eSignature solution firm. Its cloud-based platform for automating the agreement process enables more than 370,000 companies and hundreds of millions of users in over 180 countries to accelerates  business and simplify life.

DocuSign have a rich rest API interface and also have a SDK supports for many different languages like Java, C# , PHP etc. We can use these APIs to create and upload document, envelop and Templates on DocuSign server. Before going to do this, we need to setup a sandbox environment on DocuSign server. Basically, DocuSign provides a sandbox dev environment where you can test your application thoroughly. It is free of cost, so I would recommend to test your application in sandbox environment prior to the prod deployment.

Please go through this link to create a sandbox dev account in DocuSign server. After creation of an account, you would required to setup an authentication & authorization process with the DocuSign server for integration of your service with it. In DocuSign, it supports three type of authentications mechanism:

  • Authorization Code Grant: The Authorization Code Grant is an OAuth 2.0 flow that is used to grant an access token to server-hosted applications.
  • Implicit Grant: The Implicit Grant is an OAuth 2.0 flow that is used to grant an access token to user applications that are native to mobile devices.
  • JWT Grant: The JSON Web Token (JWT) grant is an OAuth 2.0 flow that is used to grant an access token to service integrations.

For more info on DocuSign authentication process. Please go through this link.

To integrate the DocuSign service to our application. I have created a sample project in Scala. So once the setup and authentication process complete, we would require to put all essential credentials in application.conf file. It looks like this.

"knoldus": {
"ds_client_id" : "",
"ds_impersonated_id": "",
"ds_auth_server": "account-d.docusign.com",
"account_id": "",
"token_expiration_in_seconds": 3600,
"ds_private_key":"",
"base_path": "https://demo.docusign.net/restapi"
}

The following keys, we would required to put in application.conf:

  • ds_client_id: It is an integration key of your registered app on DocuSign server.
  • ds_impersonated_id: It is a user id for the DocuSign APIs.
  • account_id: This is an id of your sandbox dev account.
  • ds_private_key: This is a RSA private key of your DocuSign application.

Now, we have to create a JWT token with help of the above credentials prior to upload any document or template on server. For that, we have created a Scala method.

def createAndSetJWTAcessToken(clientCredentials: DocuSignCredentials): Try[OAuth.OAuthToken] = {
    Try {
      apiClient.setOAuthBasePath(clientCredentials.dsAuthServer)
      val oAuthToken = apiClient.requestJWTUserToken(
        clientCredentials.dsClientId,
        clientCredentials.dsImpersonatedId,
        List(OAuth.Scope_SIGNATURE).asJava,
        clientCredentials.dsPrivateKey.getBytes,
        clientCredentials.tokenExpirationInSeconds)
      apiClient.setAccessToken(oAuthToken.getAccessToken,
      oAuthToken.getExpiresIn)
      apiClient.setBasePath(clientCredentials.basePath)
      oAuthToken
    }
  }

Once we got a valid JWT token then we can upload any document and DocuSign will automatically put the user or organization eSignature on uploaded document. Here we have a upload document method.

def uploadDocOnDocusign(
    clientCredentials: DocuSignCredentials,
    documentName: String,
    documentExtension: String,
    documentInputStream: InputStream): Try[DocuSignTemplateId] = {
     for {
      _ <- createAndSetJWTAcessToken(clientCredentials)
    } yield {
      val templateId = templatesApi.createTemplate(
        clientCredentials.accountId,
        createTemplate(documentName, documentExtension,
        documentInputStream)).getTemplateId
        DocuSignTemplateId(templateId)
    }
  }

You can also see the uploaded and signed documents on DocuSign server. For this you have to log in to account and go to template section to see the list of uploaded document.
The full DocuSign integration source code is available here.

Stay Tunes, happy learning  !!!

References:
https://developers.docusign.com/esign-rest-api/guides/authentication
https://developers.docusign.com/?_ga=2.63235628.2006265929.1582902232-1670025300.1582902232

MachineX: The second dimensionality reduction method

Knoldus Blogs

In the previous blog we have gone through how more data or to be precise more dimensions in the data creates different problems like overfitting in classification and regression algorithms. This is known as “curse of dimensionality”. Then we have gone through the solutions to the problem i.e. dimensionality reduction. We were mainly focused on one of the dimensionality reduction method called feature selection. In this blog, we will go through the second dimensionality reduction method that we were discussing.

Unlike feature selection, feature extraction doesn’t create the subsets and then find the best one out of them, instead, it tries to create whole new features set from the existing features. For example we have the data set X = {x1, x2, x3…..xn}, after doing the feature extraction it would be something like Y = {y1, y2, y3, ….ym} where m is the new number of dimensions extracted out. If…

View original post 960 more words

Introduction about the Trade Life Cycle

Hello folks, in this blog we will learn about trade life cycle. So before going into detail, one genuine question comes to mind is What is Trade life cycle?

In the financial market, “trade” means to buy and/or sell securities/financial products. To explain it further, a trade is the conversion of an order placed on the exchange which results in pay-in and pay-out of funds and securities. The trade ends with the settlement of the order placed. All the steps involved in a trade, from the point of order receipt (where relevant) and trade execution through to settlement of the trade, are commonly referred to as the ‘trade lifecycle’.

The Trade Life Cycle mainly divided into two parts:

  1. Trading Activity
  2. Operational Activity

Trading Activity:  Under this activity, it covers all process and procedure to capture trade from the client via front office and enrich that trade so it will able to send it for operational activity. This activity has two parts.

  • Trade Execution
  • Trade Capture (Front office)

Trade Execution: In this logical step we determine the trading business channel where sellers and buyers execute trades. On the basis of business channel trading markets classified into two categories.

  1. Quote-driven Markets: In markets where market makers publicise (quote) prices at which they are prepared to buy and sell with the intention of attracting a counterparty, the market is said to be ‘quote-driven’. Examples of quote-driven markets where quoted prices are displayed on computer screens are Nasdaq (US), SEAQ (UK) and the Eurobond market trade execution typically occurs by telephone or electronically.
  2. Order-driven Markets: In markets in which orders from sellers are compared and matched with buyers’ orders electronically, the market is said to be ‘order-driven’.

    Examples of order-driven markets are: SEATS (Australia), Xetra (Germany), SETS (UK).

Trade Capture (Front office): The successful capture of a trade within a trading system should result in the trade details being sent to the back office immediately, via an interface, for operational processing. Where an STO (Securities Trading Organisation) has no trading system (nowadays a rare circumstance), the trade detail is usually recorded manually, by the trader or market maker, onto a ‘dealing slip’ this will require collection by, or delivery to, the middle office or settlement department for operational processing. Under these circumstances, the trader or market maker will need to maintain their trading position manually, keeping it updated with any new trades.

The Pictorial representation of Trade life cycle is :

The_trade_lifecycle

In next blog, we will learn the role of operational activity in trade life cycle.

Introduction to FIX protocol for Trading

The Financial Information eXchange(FIX) protocol is an electronic communications protocol initiated in 1992 for international real-time exchange of information related to the securities transactions and markets. It was initiated by a group of institutions interested in streamlining their trading processes. These firms believed that they, and the industry as a whole, could benefit from efficiencies derived through the creation of a standard for the electronic communication of indications, orders and executions. The result is FIX, an open message standard controlled by no single entity that can be structured to match the requirements of each firm.

FIX has become the language of the global financial markets used extensively by buy and sell-side firms, trading platforms and even regulators to communicate trade information. This non-proprietary, free and open standard is constantly being developed to support evolving business and regulatory needs and is used by thousands of firms every day to complete millions of transactions.

FIX is the way the world trades and it is becoming an essential ingredient in minimizing trade costs, maximizing efficiencies and achieving increased transparency. FIX offers significant benefit to firms keen to explore new investment opportunities, it reduces the cost of market entry with participants able to quickly communicate both domestically and internationally, in addition to significantly reducing switching costs.

What FIX does for us:

  • Provides an industry standard means to communicate the information you wish to communicate electronically. This could be:
    • Indications of Interest
    • Orders and Order Acknowledgment
    • Fills
    • Account Allocations
    • News, E-Mail, Program Trading Lists
    • Administrative Messages
  • Connects us with an expanding list of buy and sell-side institutions
  • Delivers information in real-time

 The Following type of securities we can process via FIX:

  • US Equities
  • International Equities (all markets)
  • ADRs
  • Convertible Bonds
  • Futures
  • Foreign Exchange trading
  • Fixed Income

Pictorial representation of FIX system:

The given diagram shows how FIX messaging looks between Buyside/Customer and Sellside/Supplier.

Financial_Information_eXchange_System_Connectivity_Diagram

FIX message format layout:

The FIX message is a combination of header, body, and trailer. That means header + body + trailer = FIX content.
Up to FIX.4.4, the header contained three mandatory fields: 8 (BeginString), 9 (BodyLength), and 35 (MsgType) tags.

Example of a FIX message: Execution Report (Pipe character is used to represent SOH character)

8=FIX.4.2 | 9=178 | 35=8 | 49=PHLX | 56=PERS | 52=20071123-05:30:00.000 | 11=ATOMNOCCC9990900 | 20=3 | 150=E | 39=E | 55=MSFT | 167=CS | 54=1 | 38=15 | 40=2 | 44=15 | 58=PHLX EQUITY TESTING | 59=0 | 47=C | 32=0 | 31=0 | 151=15 | 14=0 | 6=0 | 10=128 |

Body Length:
Body length is the character count starting at tag 35 (included) all the way to tag 10 (excluded) which is for checksum. SOH delimiters do count in body length. Tag 9 represents body length. In the above example body length of this message is 178.

Checksum:
The checksum algorithm of FIX consists of summing up the decimal value of the ASCII representation all the bytes up to but not including the checksum field (which is last) and return the value modulo 256. Tag 10 is used for checksum value, in the above example checksum value is 128.

In a nutshell, we can say that FIX standard is a unified stack for all trading related messaging process. We can use FIX to send order, allocation, allocation instruction and its execution related message. It also supports confirmation, acceptance and rejection type of acknowledgment messages to send a report back to the client about his order’s execution status.

To get more information about FIX, you can visit the official site.


knoldus-advt-sticker


Knoldus Bags the Prestigious Huawei Partner of the Year Award

Knoldus Blogs

Knoldus was humbled to receive the prestigious partner of the year award from Huawei at a recently held ceremony in Bangalore, India.

huawei-knoldus-award

It means a lot for us and is a validation of the quality and focus that we put on the Scala and Spark Ecosystem. Huawei recognized Knoldus for the expertise in Scala and Spark along with the excellent software development process practices under the Knolway™ umbrella. Knolway™ is the Knoldus Way of developing software which we have curated and refined over the past 6 years of developing Reactive and Big Data products.

Our heartiest thanks to Mr. V.Gupta, Mr. Vadiraj and Mr. Raghunandan for this honor.

att00001

About Huawei

Huawei is a leading global information and communications technology (ICT) solutions provider. Driven by responsible operations, ongoing innovation, and open collaboration, we have established a competitive ICT portfolio of end-to-end solutions in telecom and enterprise networks, devices, and cloud computing…

View original post 170 more words

Transaction Management in Cassandra

Knoldus Blogs

As we are all from the Sql Background and its been ages SQL rules the market , so transaction are something favorite to us .
While Cassandra does not support ACID (transaction) properties but it gives you the ‘AID’ among it .
That is Writes to Cassandra are atomic, isolated, and durable in nature. The “C” of ACID—consistency—does not apply to Cassandra, as there is no concept of referential integrity or foreign keys.

Cassandra offers you to tune your Consistency Level as per your needs . You can either have partial or full consistency that is You might want a particular request to complete if just one node responds, or you might want to wait until all nodes respond .

We will talk here about the ways we can implement the so called transaction concept in Cassandra .

Light Weight Transactions
They are also known as compare and set transactions

View original post 777 more words

Twitter’s tweets analysis using Lambda Architecture

Hello Folks,

In this blog i will explain  twitter’s tweets analysis with lambda architecture. So first we need to understand  what is lambda architecture,about its component and usage.

According to Wikipedia, Lambda architecture is a data processing architecture designed to handle massive quantities of data by taking advantage of both batch and stream processing methods.

Now let us see  lambda architecture components and its detail.

lambda-architecture-2-800

This architecture is classified into three layer :

Batch Layer : The batch layer precomputes the master dataset(it is core components of lambda architecture and it contains all data) into batch views so that queries can be resolved with low latency.

Speed Layer: In speed layer we are basically do two things,storing the realtime views and processing the incoming data stream so as to update those views.It fills the delta gap that is left by batch layer.that means combine speed layer view and batch view give us capability fire any adhoc query over all data that is query=function(over all data).

Serving Layer: It provides low-latency access to the results of calculations performed on the master dataset . It combines batch view and realtime view to give result in realtime for any adhoc query over all data.

So in short we can say lambda architecture is query=function(over all data).

Now i am going to describe twitter’s tweet analysis with the help of lambda architecture.This project uses twitter4j streaming api and Apache Kafka  to get and store twitter’s realtime data.I have used Apache Cassandra  for storing Master dataset ,batch view and realtime view.

Batch Layer of project :  To process data in batch we have used Apache Spark  (fast and general engine for large-scale data processing) engine and to store batch view we have used  cassandra. To do this we have created BatchProcessingUnit to create all batch view on master dataset.

class BatchProcessingUnit {

  val sparkConf = new SparkConf()
    .setAppName("Lambda_Batch_Processor").setMaster("local[2]")
    .set("spark.cassandra.connection.host", "127.0.0.1")
    .set("spark.cassandra.auth.username", "cassandra")

  val sc = new SparkContext(sparkConf)

  def start: Unit ={
    val rdd = sc.cassandraTable("master_dataset", "tweets")
    val result = rdd.select("userid","createdat","friendscount").where("friendsCount > ?", 500)
    result.saveToCassandra("batch_view","friendcountview",SomeColumns("userid","createdat","friendscount"))
    result.foreach(println)
  }
}

We have used Akka scheduler to schedule batch process in specified interval.

Speed Layer of project: In speed layer we have used Spark Streaming  to process realtime tweets and store its view in cassandra.

To do this we have created SparkStreamingKafkaConsumer which read data from kafka queue “tweets” topic and send to view handler of speed layer to generate all view.

object SparkStreamingKafkaConsumer extends App {
  val brokers = "localhost:9092"
  val sparkConf = new SparkConf().setAppName("KafkaDirectStreaming").setMaster("local[2]")
    .set("spark.cassandra.connection.host", "127.0.0.1")
    .set("spark.cassandra.auth.username", "cassandra")
  val ssc = new StreamingContext(sparkConf, Seconds(10))
  ssc.checkpoint("checkpointDir")
  val topicsSet = Set("tweets")
  val kafkaParams = Map[String, String]("metadata.broker.list" -> brokers, "group.id" -> "spark_streaming")
  val messages: InputDStream[(String, String)] = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topicsSet)
  val tweets: DStream[String] = messages.map { case (key, message) => message }
  ViewHandler.createAllView(ssc.sparkContext, tweets)
  ssc.start()
  ssc.awaitTermination()
}

Serving Layer of Project: In serving layer basically we have combined batch view data and realtime view data to satisfy adhoc query requirement.Here is an example in which have try to analyse all twitter users who match the specify hashtag and they have follower counts greater than 500 .

def findTwitterUsers(minute: Long, second: Long, tableName: String = "tweets"): Response = {
  val batchInterval = System.currentTimeMillis() - minute * 60 * 1000
  val realTimeInterval = System.currentTimeMillis() - second * 1000
  val batchViewResult = cassandraConn.execute(s"select * from batch_view.friendcountview where createdat >= ${batchInterval} allow filtering;").all().toList
  val realTimeViewResult = cassandraConn.execute(s"select * from realtime_view.friendcountview where createdat >= ${realTimeInterval} allow filtering;").all().toList
  val twitterUsers: ListBuffer[TwitterUser] = ListBuffer()
  batchViewResult.map { row =>
    twitterUsers += TwitterUser(row.getLong("userid"), new Date(row.getLong("createdat")), row.getLong("friendscount"))
  }
  realTimeViewResult.map { row =>
    twitterUsers += TwitterUser(row.getLong("userid"), new Date(row.getLong("createdat")), row.getLong("friendscount"))
  }
  Response(twitterUsers.length, twitterUsers.toList)
}

Finally this project used Akka HTTP for build rest api to  fire adhoc queries.

I hope, it will be helpful for you in building  Big data application using lambda architecture.

You can get source code here

References:

 https://en.wikipedia.org/wiki/Lambda_architecture

http://lambda-architecture.net/

https://www.mapr.com/developercentral/lambda-architecture

 


KNOLDUS-advt-sticker