- Experience in software development 5+ years
- Experience in Big Data
- Experience with Build Services
- Spark/Spark streaming, Map reduce, Flink
- Java OR Scala
WOULD BE BENEFICIAL:
- Data stream like NSQ, kafka etc.
- Protobuf, avro etc.
- Parquet, RDD, sequence files
- Data warehouse DB like Snowflakes, Redshift
- Spring (Boot, MVC, Security)
- Micro Service
- Cloud (AWS etc.)
- Object store (S3 etc.)
- Data flow engines (Airflow, Ozzie,…)
- Automated (unit) testing big data
- Big data visualisation
- Etcd, zookeeper
Create scalable efficient low-latency advanced processing of big data which get used for reporting and analytics in data warehouse such as Snowflake.
Provide needed micro services to get the overall data flow working and allow the portal to access the produced data.
CyberCraft on behalf of Silicon Valley based startup is looking for a sharp Senior Big Data Engineer to help build an Ad Tech solution that speeds up your internet. The solution balances requests that websites cannot control due to complex advertisement solutions and controls all requests the page receives. You will work on reducing latency by applying various technologies for speeding up downloads and on eliminating the need for third-party DNS lookups, SSL handshakes, etc