Big Data Engineer

Job expired!
Codete is not just a software company, it's a place where tech enthusiasts can grow by doing what they love and feel valued for who they are. We're experienced, agile, and versatile: we work with a wide range of technologies in projects from various industries, and the majority of our team are senior-level specialists. Together with our technological partner, we're looking for a Big Data Engineer to help us create an advanced search engine tailored for business data across different industries. We achieve this by enhancing business data with an unrivaled focus on small and medium-sized enterprises (SMBs), ensuring precise categorization, and offering deep insights powered by real-time information. Requirements: - Experience using statistical computer languages, preferably Scala (or R, Python, SQL, etc.) to manipulate data and draw insights from very large datasets. - Experience working with and creating data architectures (Spark, Cassandra). - Knowledge of advanced statistical techniques and concepts (regression, properties of distributions, statistical tests and proper usage, etc.) and experience with applications. - Any knowledge of machine learning techniques (clustering, decision tree learning, artificial neural networks, etc.) and their real-world advantages/disadvantages is a big plus. Challenges: - Mine and analyze data from across the web. - Assess the effectiveness and accuracy of new data sources and data gathering techniques. - Develop custom data models and algorithms to apply to text datasets. - Use predictive modeling to increase and optimize data extraction and data quality at ingestion and post-processing. - Develop company's A/B data testing framework and test data quality continuously. - Develop processes and tools to monitor and analyze model performance and data accuracy. - Prototype quickly to solve thorny use cases, without getting stuck in theory, as we're prone to shipping early and often. - Write well-designed, testable, efficient code. - Identify areas of opportunity and improvement. Location: Krakow, Lublin, remote Salary: B2B: 150 -170 PLN/h