Embedded Analytics: Scoot Science and the Universal Semantic Layer

Transforming ocean data into actionable insights

Founded in 2017, Scoot Science has swiftly emerged as a leader in ocean observation and marine technology. Specializing in oceanography and ocean-related data science, Scoot Science leverages flexible embedded analytics, and oceanographic modeling to help marine operators manage ocean risk.

At the heart of Scoot Science's innovation is SeaState, an embedded analytics solution that serves as a single pane of glass for fish farms. This powerful virtual representation of the farms integrates data from global IoT sensors and various fish management platforms, providing a unified view of critical metrics such as water temperature, oxygen saturation, and salinity. Based on the data collected, they are even able to forecast conditions up to ten days out, just like the weather. By offering real-time insights and comprehensive analytics, Scoot Science enables clients who run fish farms to make informed decisions and respond proactively to environmental changes.

Caption: Scoot Science SeaState dashboard platform powered by Cube Cloud

Overcoming data complexity and performance challenges

Before partnering with Cube, Scoot Science faced significant challenges with their data stack complexity and inconsistent business definitions. Previously, the team managed data aggregation and standardization through raw SQL queries within a GraphQL API framework. For instance, obtaining the average water temperature by the hour over a week required extensive manipulation of graph endpoints or custom SQL queries. Slight variations in data requirements meant revisiting and updating SQL queries, leading to inefficiencies and performance bottlenecks.

Scoot Science's infrastructure struggled to maintain database performance amidst the volume and variety of oceanographic data streams they needed to analyze. The absence of advanced caching mechanisms exacerbated these issues, making it difficult to provide timely insights to clients. The complexity of managing numerous metrics and data streams resulted in a fragmented approach to data analytics, hindering the team's ability to deliver consistent and reliable results.

The team realized that continuing with their current approach would only compound the performance issues. They needed a solution that could handle the increasing data load while providing flexibility and efficiency. This search for a robust data management solution led them to Cube, marking the beginning of a series of transformative innovations for Scoot Science and its clients.

Caption: Scoot Science data stack

Transitioning from Cube Core to Cube Cloud

Initially, Scoot Science adopted the open-source version of Cube, running it for six to eight months. While Joshua Hubers, the Lead Software Engineer at Scoot Science, described Cube Core as “sophisticated, powerful, and flexible”, its complexity and operational overhead presented challenges for a small team. He found himself juggling multiple roles and responsibilities.

Joshua explained, “We're a small team. I'm pretty much, at this point, the only maintainer of Cube at Scoot Science. I just have so many other roles and responsibilities to maintain. Our infrastructure needs are pretty big and there's a lot of moving pieces to the Cube system. There was a lot of strain on my hours because of the upkeep to maintain the system. And so one of the biggest benefits (to switching) was that I didn't have to worry about that anymore.”

Joshua notes that the open-source version of Cube was valuable, offering a great foundation, but the transition to Cube Cloud was driven by the need to reduce maintenance burdens and streamline operations. The transition to Cube Cloud allowed Scoot Science to fully leverage flexible embedded analytics, reducing the burden of infrastructure maintenance.

Enhancing efficiency and flexibility with embedded analytics

Migrating to Cube Cloud brought numerous benefits to Scoot Science, significantly enhancing their data management, embedded analytics capabilities, and overall operational efficiency. Not only did Scoot Science implement flexible embedded analytics solutions to streamline data aggregation and improve performance, but Joshua also expressed his appreciation for the additional features and user-friendly interface that Cube Cloud offers. "After the migration, we've definitely loved all the extra Cube Cloud UI benefits that come with it. We really enjoy being able to poke around all the pre-aggregations through the views. That didn't come with the open-source version."

Advanced monitoring and support

One of the standout advantages of Cube Cloud for Scoot Science is its robust monitoring capabilities. "I've really appreciated all the monitoring work. Live query monitoring and SQL monitoring are really nice. That was something we didn't have before and just didn't have time to implement," Joshua noted. The ability to rely on Cube Cloud to manage these aspects allowed the team to focus on other critical tasks. "It's nice that you could take the couple of queries that you've written and then just throw it out to the cloud and say, 'manage it for me.' That's been the nicest thing so far."

Furthermore, the Cube team played a significant role in the migration process, offering continuous support and suggestions to optimize performance. "The team at Cube was very gracious, providing numerous suggestions to speed things up. They recommended specific implementations, ways to address our queries, work around the complexity of multi-tenancy, and strategies for building aggregations. I was really grateful for the hours and all the advice they provided to get us up to speed. (Cube Cloud) was even faster than it was with the open-source version," Joshua said. This collaborative effort ensured that Scoot Science's application was optimized for the shared cloud environment.

Powerful data management features

Another benefit of Cube Cloud is that it powers high-level metrics and summaries, allowing Scoot Science to manage extensive time series data with precision. This capability is essential for their unique use cases, such as the alarm feature and the custom chart builder. The alarm system built using Cube, that many Scoot Science clients use, notifies fish farms of extreme ocean conditions such as low oxygen levels. This feature is highly flexible, enabling clients to pull data screens and set up customized alerts.

Caption: Demo of Scoot Science’s unique alarm feature built using Cube’s technology

The custom chart builder, another innovative feature, allows clients to create personalized charts and graphs based on their data. This functionality is particularly valuable for clients investing in mitigation strategies, such as aeration systems.

Caption: Demo of Scoot Science’s custom chart builder feature built using Cube’s technology

Updating the Cube data model to include new metrics is a straightforward and efficient process. Joshua emphasized the ease of using the Cube declarative data models. "People were interested in the weather metrics of their site, so we put a rain gauge, updated the database to include information, then updated the Cube schema and wrote a query. All in all, the process of building, testing, and debugging takes maybe half a day or a day. It’s really easy and lightweight, especially with the Cube framework."

Freed up time for innovation

One of the most significant benefits of Cube Cloud is the time it has freed up for Joshua and his team. "I finally have time to write code again instead of spending all my time managing infrastructure. It’s nice to say that I can write code again." The structured semantic layer provided by Cube has streamlined their data querying process. "Having the semantic layer with its hammered-out conventions is super nice because it's not a question anymore of ‘how do I query this data?’ Cube's answered those questions for us, so I’m just writing a bit of SQL for this one use case, if it's even that far down the hole. Oftentimes you only have to write one line of SQL, and it's good."

Caching and pre-aggregations have made a significant impact on performance, particularly for long-time series data. Joshua highlighted the efficiency gains, stating, "We aggregate nine days' worth of data by the hour, and the fact that we can wrap a precalculated pre-aggregation around this is the world. We get so many compliments, and people just gawk at it. The pre-aggregation system works, it's really powerful, and it alleviates a lot of performance issues."

Business outcomes: Driving sustainability in aquaculture

Customers greatly appreciate Scoot Science's ability to standardize data, regardless of the equipment used. This standardization means that the diverse data collected from various sources is ingested and formatted uniformly, providing consistent and reliable insights.

By leveraging flexible embedded analytics through Cube Cloud, Scoot Science has been able to enhance their data management processes, leading to better decision-making and more effective sustainability practices in the aquaculture industry. The seamless integration and powerful data capabilities of Cube Cloud have enabled Scoot Science to deliver high-quality, actionable insights to their clients, ultimately contributing to the sustainability of the global food chain through improved aquaculture practices.

Looking forward: Powering the future of Scoot Science with AI

As Scoot Science looks to the future, they are committed to harnessing the power of AI to further enhance their platform and user experience. By joining the growing wave of AI integration, Scoot Science aims to transform how users interact with and interpret oceanographic data. The team is exploring the potential of plugging an AI model, specifically a Large Language Model (LLM), into their platform to simplify data analysis and interpretation.

This approach works perfectly with Cube's AI API. With Cube's AI API, Scoot Science can explore features like simple language summaries such as "This is what your site is looking like this week" or "typical ocean conditions." This would make complex data more accessible and understandable for the average user, enhancing decision-making processes and proactive management in aquaculture.

Leveraging Cube's newest AI capabilities, Scoot Science aims to create a more intuitive, user-friendly, and engaging experience for their clients, reinforcing their leadership in marine technology and sustainable aquaculture.