Big Data Developer
20 Queen St West 24th Floor Toronto, ON M5H 3R3
Excellent opportunity within Data Science and Innovation group of our Financial services client.
Initially a 6 month assignment with strong potential for renewal.
A successful big data engineer within RMA/DSMI, will:
· MS in Computer Science, Statistics, Applied Mathematics, or a related field; or BS and 3+ years of relevant work experience Experience with a modern Big Data processing stack including Apache Spark, Storm, Kafka, Kinesis or equivalent technologies
· Proficiency with UNIX development environments Excellent programming skills in Python or Java
· Strong knowledge of all traditional Data Warehouse-related components (Sourcing, ETL, Data Modeling, Infrastructure, BI, Reporting) and the modern tools to support those components
· Strong SQL abilities and experience with massive relational database systems. Familiarity with the MPP Databases (Redshift or Vertica), Hadoop ecosystem, and HQL a plus.
· Work with configuration management tools in order to manage the infrastructure via code(Puppet / Ansible / Chef)
· Work with container solutions for blue green deployment (Docker)
· Work on deployment pipeline on external cloud platforms (AWS / AZURE / GCP)
· Work with continuous integration and continuous deployment (CI/CD)
· Strong foundation in Software Engineering and Core Computer Science (including data structures and algorithms)
· Highly motivated, very well organized, task oriented with leadership skills
· Strong communication with ability to gap the bridge between Business and IT
· Experience with Python, Java, Git, Hadoop, Spark, Hive, Beeline
· Experience with bash scripting and REST API