Originally published on August 23, 2021.
Application security is usually done with a set of tools and services known as SIEM – Security Information and Event Management. SIEM tools usually try to provide visibility into an organization’s security systems, as well as event log management and security event notifications.
The company Panther takes traditional SIEM security a step further. Panther processes and retains all of your security data with cloud-first workflows, identifies and alerts in real-time suspicious activity, enables building a high-fidelity alerting pipeline with Python, version control, unit tests, and CI/CD, and provides a security data lake where raw logs are structured for security at scale.
Sponsorship inquiries: firstname.lastname@example.org
Cloud computing provides tools, storage, servers, and software products through the internet. Securing these resources is a constant process for companies deploying new code to their cloud environments. It’s easy to overlook security flaws because company applications are very complex and many people work together to develop them. Wyze Labs, for example, had millions of The post Bridgecrew: Cloud Security with Guy Eisenkot appeared first on Software Engineering Daily.
20 September 2022 •
18 September 2022 •
Software engineering can be a surprisingly grueling career. It is both physically and mentally demanding to sit in front of a screen for hours on end producing code. Andy Johns is our guest in this episode. Andy previously worked on growth at Facebook, Twitter, and Quora as an early employee. He’s also an investor in The post Mental Health in Tech with Andy Johns appeared first on Software Engineering Daily.
15 September 2022 •
Data analytics technology and tools have seen significant improvements in the past decade. But, it can still take weeks to prototype, build and deploy new transformations and deployments, usually requiring considerable engineering resources. Plus, most data isn’t real-time. Instead, most of it is still batch-processed. Tinybird Analytics provides an easy way to ingest and query The post Serverless Clickhouse for Developers with Jorge Sancha appeared first on Software Engineering Daily.
12 September 2022 •
Maintaining availability in a modern digital application is critical to keeping your application operating and available and to keep meeting your customers growing demands. There are many observability platforms out there and certainly Prometheus is a popular open source solution for cloud native companies yet operating an observability platform, costs money, and all of the The post Cloud Native Observability with Martin Mao appeared first on Software Engineering Daily.
31 August 2022 •
As software engineering teams start to build products that become more and more mature, it becomes necessary to be able to debug complex issues with tooling that enables understanding of the full scenario. This can come from application scenarios where APIs are running multiple versions, where users are using your app from multiple devices, device The post Developer-first Observability with Liran Haimovitch appeared first on Software Engineering Daily.
31 August 2022 •
Web infrastructure has evolved from individual servers to shared hosting services to virtual machines and virtual functions. The future of the internet however is looking toward a much more distributed computation model . Blockchain technology is central to the future of this modern internet . Blockchains are still in their infancy. And the most people The post Modular Blockchain Architecture with Nader Dabit appeared first on Software Engineering Daily.
30 August 2022 •
Originally published on April 12, 2022. As companies move to Spark and a Lakehouse architecture, they are realizing that the data tools are lagging way behind. You need to be a programmer to effectively use Spark and Airflow. There are some low-code ETL tools, but is that enough? Companies want to treat their data pipelines The post Lakehouse Data Stack with Raj Bains appeared first on Software Engineering Daily.
25 August 2022 •
Originally published on February 12, 2022. Lee Atchison spent seven years at Amazon working in retail, software distribution, and Amazon Web Services. He then moved to New Relic, where he has spent four years scaling the company’s internal architecture. From his decade of experience at fast-growing web technology companies, Lee has written the book Architecting The post Architecting for Scale with Lee Atchison appeared first on Software Engineering Daily.
24 August 2022 •
Data is becoming a bank’s biggest asset. These complex enterprises have a huge opportunity ahead – to transform themselves to become a trusted hub of a much broader data ecosystem that goes beyond the financial industry and helps to form a new class of cross-industry experience architectures that are scalable and transparent. The data physics The post Data Infrastructure for Finance appeared first on Software Engineering Daily.
18 August 2022 •
Cloud native applications utilizing microservice architectures has grown into one of the most popular application architectural patterns in recent years. The value of leveraging dynamic cloud resources, along with the flexibility and scalability of microservice architectures, creates a strong paradigm that’s hard to miss. The strong adoption of Kubernetes has strengthened the pattern enormously. The The post K8s Troubleshooting with Itiel Shwartz appeared first on Software Engineering Daily.
16 August 2022 •
Permissions are Hard! And they are becoming harder as we move more into the Cloud-native ecosystem. If we go back in time to the point where it was just a single monolith that you were building on your own. You’ll probably have a framework to manage the permissions for you. But when you are working The post Permissions and Access-control with Or Weis appeared first on Software Engineering Daily.
16 August 2022 •
Enabling authorization policies across disparate cloud-native environments such as containers, microservices and modern application delivery infrastructure is complex and can be a roadblock for software engineering teams. Open Policy Agent, or OPA, is an open, declarative, policy-as-code approach to authorization that reduces security and compliance burden for engineering teams. Business context is translated into declarative The post Cloud-native Authorization with Tim Hinrichs appeared first on Software Engineering Daily.
9 August 2022 •
Ian Coe CEO Adam Kamor Head of Engineering Companies that gather data about their users have an ethical obligation and legal responsibility to protect the personally identifiable information in their dataset. Ideally, developers working on a software application wouldn’t need access to production data. Yet without high-quality example data, many technology groups stumble on avoidable The post Faking Data Using Tonic.ai with Ian Coe and Adam Kamor appeared first on Software Engineering Daily.
5 August 2022 •
Couchbase is a distributed NoSQL cloud database. Since its creation, Couchbase has expanded into edge computing, application services, and most recently, a database-as-a-service called Capella. Couchbase started as an in-memory cache and needed to be rearchitected to be a persistent storage system. In this episode, We interviewed Ravi Mayuram, SVP Products, and Engineering at Couchbase. The post Couchbase with Ravi Mayuram appeared first on Software Engineering Daily.
28 July 2022 •
Originally published on April 12, 2016. Negotiation is an important skill for software engineers. The salary you negotiate at the beginning of your job could be a difference of tens of thousands of dollars over the course of an engineer’s career, but intimidating recruiters and exploding offers scare many engineers from negotiating at all. Today, The post Salary Negotiation with Haseeb Qureshi appeared first on Software Engineering Daily.
23 July 2022 •
Remix is a full stack web framework that lets you focus on the user interface and work back through web fundamentals to deliver a fast, slick, and resilient user experience that deploys to any Node.js server and even non-Node.js environments at the edge like Cloudflare Workers. In this episode, we interviewed Ryan Florence, co-founder at The post Remix with Ryan Florence appeared first on Software Engineering Daily.
22 July 2022 •
Profiling has been a part of the software engineering toolbox since the 1970s, but it was long regarded to be too high in overhead, so it was only done ad hoc. The problem with profiling that way is that you never catch the moment in time where for example an incident happened. Continuous profiling is The post Continuous Profiling using eBPF with Frederic Branczyk appeared first on Software Engineering Daily.
16 July 2022 •
Enterprise-grade authentication is often an essential ingredient to virtually all applications in today’s world. However, companies often have a hard time understanding the value of that authentication especially during the early stages of product development. And hardening of an application is often left as an afterthought. Add enterprise-level requirements such as single sign-on and two-factor The post WorkOS with Michael Grinich appeared first on Software Engineering Daily.
6 July 2022 •
Build automation tools automate the process of building code, including steps such as compiling, packaging binary code, and running automated tests. Because of this, build automation tools are considered a key part of a continuous delivery pipeline. Build automation tools read build scripts to define how they should perform a build. Common build scripts include The post Earthly Builds with Adam Gordon Bell appeared first on Software Engineering Daily.
1 July 2022 •
Creating great creative tools is extremely difficult. There are thousands of paths a user could take, and every aspect of the user experience has to be carefully considered and optimized for performance. And when you try to add real-time collaboration on top of that, you can be sure that you’re left with almost no time The post LiveBlocks with Guillaume Salles appeared first on Software Engineering Daily.
30 June 2022 •
Most technology companies rely on open source software projects. But open source software projects are often maintained by a group of people that is not affiliated with any particular company. When an open source project develops too much technical debt, it can become a tragedy of the commons. Who is responsible for maintaining these open The post Gitcoin Ecosystem with Gloria Kimbwala appeared first on Software Engineering Daily.
30 June 2022 •
Mailchimp is an all-in-one marketing platform for growing businesses, empowering millions of customers around the world to launch, build, and grow their businesses with world-class marketing technology, award-winning customer support, and inspiring content. Eric Muntz is Mailchimp’s CTO, responsible for the engineering teams that design, implement, and maintain Mailchimp’s products and infrastructure. He joins the The post Mailchimp Engineering with Eric Muntz appeared first on Software Engineering Daily.
28 June 2022 •
Venture capital investment has continued to flow into technology startups. No one builds technology from scratch. There are cloud services, software libraries, 3rd party services, and software platforms that modern entrepreneurs must adopt to build their products efficiently and quickly. These layers of infrastructure are a key area for many investors. Tim is a partner The post Technical Venture Capital with Tim Tully appeared first on Software Engineering Daily.
27 June 2022 •
Between hyper financialisation and the growth of automation in the world, trade and investments have become an ever more prevalent and important tool for the mass market. But the relative knowledge of professional and hobbyist investors is different, and so is the access to opportunities. Coinrule empowers regular people to compete with professional traders by The post Coinrule Automated Crypto Trading with Zdenek Hofler appeared first on Software Engineering Daily.
25 June 2022 •
Breakthroughs from labs that are exclusively or mostly focused on research tend to stay buried as papers. Sometimes the lag is natural, the research is far ahead of its broad applicability. But we find that there are hundreds of breakthroughs useful long before they reach users. It can take many years, even decades, before breakthroughs The post Protocol Labs Research with Jeromy Johnson appeared first on Software Engineering Daily.
24 June 2022 •
As crypto gains worldwide attention, the most scalable, resilient and performant systems are required for the industry to thrive. Products must be designed for a mix of technical and non-technical users. Blockchain.com’s VP of Engineering Lewis Tuff takes us behind the scenes for a look at the architecture, programming languages and database choices required to The post Building for Crypto with Lewis Tuff appeared first on Software Engineering Daily.
22 June 2022 •
This episode is hosted by Lee Atchison. Lee Atchison is a software architect, author, and thought leader on cloud computing and application modernization. His most recent book, Architecting for Scale (O’Reilly Media) is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host The post Kubernetes Spend with Webb Brown appeared first on Software Engineering Daily.
17 June 2022 •
Web3 promises to change the way we interact and transact online, but today’s dapps and wallets are hard to use. They are typically optimized for desktop machines and domain experts. In particular, it can be hard to bring value on-chain, and hard to know what to do once you’ve onboarded. Charlie Andrews-Jubelt is a software The post Valora Decentralized Venmo with Charlie Andrews-Jubelt appeared first on Software Engineering Daily.
16 June 2022 •
While Kubernetes has many benefits, there is often a need for teams to deploy a monitoring and observability stack to troubleshoot issues that happen within the cluster and the applications themselves. ContainIQ, an out-of-the-box solution, allows engineers to monitor the health of their cluster and troubleshoot issues faster. ContainIQ is unique in its approach in The post ContainIQ with Matthew Lenhard appeared first on Software Engineering Daily.
14 June 2022 •
This episode is hosted by Alex DeBrie. Alex is the author of The DynamoDB Book, the comprehensive guide to data modeling with DynamoDB, as well as The DynamoDB Guide, a free guided introduction to DynamoDB. He runs a consulting company where he assists clients with DynamoDB data modeling, serverless architectures, and general AWS usage. You can find The post Open-Source Cloud Asset Management with Yevgeny Pats appeared first on Software Engineering Daily.
11 June 2022 •
How to build a rocket ship while flying it? In this episode, Cos returns to the show to discuss the technologies and approach Brex chose to create an engineering organization and innovation engine for the long term, while navigating shifts as the company continued to scale at a rapid pace. Cos and Jeff also cover The post CTO Decisions with Cos Nicolaescu appeared first on Software Engineering Daily.
9 June 2022 •
Convex makes storing your application’s data as easy as using React state management. If you can use React hooks, you can also manage your backend data using Convex. James Cowling is a former Dropbox infrastructure engineer turned startup founder. James joins the show to discuss how Convex offers a simpler full-stack developer experience than the The post Global State Management with James Cowling appeared first on Software Engineering Daily.
2 June 2022 •
Developers looking for read or write access to Ethereum, Polygon, IPFS or other Web3 networks in order to get their idea in the hands of users need reliable RPC endpoints they can count on, whether they are working on a hackathon project or running hybrid architecture for a production application. Infura provides a platform for The post Infura Ethereum Infrastructure with E.G. Galano appeared first on Software Engineering Daily.
2 June 2022 •
Streaming data platforms like Kafka, Pulsar, and Kinesis are now common in mainstream enterprise architectures, providing low-latency real-time messaging for analytics and applications. However, stream processing – the act of filtering, transforming, or analyzing the data inside the messages – is still an exercise left to the receiving microservice or datastore, a custom programming exercise The post Decodable Streaming with Eric Sammer appeared first on Software Engineering Daily.
1 June 2022 •
The Kubernetes ecosystem has drastically changed how development teams ship software. While Kubernetes has provided many advancements in cloud infrastructure, it has also left organizations with massive security blindspots. KSOC was created to give developers and security teams a single control plane to harden multi-cluster Kubernetes environments through event-driven analysis, least privilege enforcement, and remediation-as-code. The post Kubernetes Security Compliance with Jimmy Mesta appeared first on Software Engineering Daily.
1 June 2022 •
In software engineering, telemetry is the data that is collected about your applications. Unlike logging, which is used in the development of apps to pinpoint errors and code flows, telemetry data includes all operational data including logs, metrics, events, traces, usage, and other analytical data. Companies usually visualize this information to troubleshoot problems and understand The post New Relic Architecture with Nic Benders appeared first on Software Engineering Daily.
28 May 2022 •
One of the challenges with Microservices architecture is how you manage dependencies between your services when implementing workflows. Conductor is an open-source microservices and workflow orchestration platform. Boney Sekh co-founded Orkes Inc – a company focused on offering Conductor as a service. Boney joins the show to discuss how engineers leverage Conductor to build highly The post Conductor Orchestration with Boney Sekh appeared first on Software Engineering Daily.
26 May 2022 •
Notifications have typically been an area of a product where building in-house has been the only option. However, building a best-in-class notifications system that delivers a great customer experience requires a significant investment when you start to layer in complexities like batching, cross channel orchestration, and user preferences. Chris Bell of Knock joins the show The post Notifications Infrastructure with Chris Bell appeared first on Software Engineering Daily.
25 May 2022 •
The advent of the cloud introduced a new form of technical debt in which organizations can lose track of what infrastructure they have and how it relates to the business. While the cloud’s native APIs offer some transparency into your infrastructure, these offerings are often described as necessary but not sufficient. When companies have a The post CloudGraph with Tyson Kunovsky appeared first on Software Engineering Daily.
21 May 2022 •
Highlight is a tool that helps teams reproduce end-to-end user sessions to troubleshoot their applications faster, more efficiently, and with all the context they need. With Highlight, engineering teams can replay errors with high precision, which includes complete session replay, outgoing network requests, dense stack traces and insight into the app’s state management system. Jay The post Browser Observability With Jay Khatri appeared first on Software Engineering Daily.
20 May 2022 •
Lee Atchison spent seven years at Amazon working in retail, software distribution and Amazon Web Services. He then moved to New Relic, where he spent eight years scaling the company’s internal architecture. Lee has deep expertise in building and managing fast growing web applications, and he used this knowledge to write architecting for scale from The post Technical Debt With Lee Atchison appeared first on Software Engineering Daily.
18 May 2022 •
There’s an increasing regulatory and consumer pressure on companies to do a better job protecting sensitive customer data. Yet, despite this pressure, data breaches and compliance issues continue to plague the tech industry. Companies like Apple, Netflix, and Google have solved these challenges by pioneering a new type of technology, the data privacy vault. Skyflow The post Skyflow Privacy and Compliance with Sean Falconer appeared first on Software Engineering Daily.
17 May 2022 •
Data-as-a-service is a company category type that is not as common as API-as-a-service, software-as-a-service, or platform-as-a-service. In order to vend data, a data-as-a-service provider needs to define how that data will be priced, stored, and delivered to users: streaming over an API or served via static files. Naqeeb Memon of Safegraph joins the show The post Data Delivery with Naqeeb Memon appeared first on Software Engineering Daily.
14 May 2022 •
Blockchains were the first systems to allow guaranteed permanent storage of public data. As cryptocurrency technology has advanced, a rich ecosystem of permanent storage and compute has developed as well. One of these is Arweave, a system for permanent information storage. Sam Williams is a founder of Arweave and joins the show to talk through The post Arweave with Sam Williams appeared first on Software Engineering Daily.
13 May 2022 •
Data labeling allows machine learning algorithms to find patterns among the data. There are a variety of data labeling platforms that enable humans to apply labels to this data and ready it for algorithms. Heartex is a data labeling platform with an open source core. Michael Malyuk joins the show to talk through the platform The post Data Labeling with Michael Malyuk appeared first on Software Engineering Daily.
11 May 2022 •
Real-time analytics are difficult to achieve because large amounts of data must be integrated into a data set as that data streams in. As the world moved from batch analytics powered by Hadoop into a norm of “real-time” analytics, a variety of open source systems emerged. One of these was Apache Pinot. StarTree is a The post Pinot and StarTree with Chinmay Soman appeared first on Software Engineering Daily.
9 May 2022 •
Web3 is powerful but difficult to work with. Deploying blockchain nodes, accessing data, and performing staking operations are non-trivial engineering actions. To simplify web3, Ankr hosts APIs for node deployment, RPC, and staking. Josh Neuroth from Ankr joins the show to talk about modern web3 infrastructure. Sponsorship inquiries: email@example.com The post Web3 Infrastructure with Josh Neuroth appeared first on Software Engineering Daily.
7 May 2022 •
Brendan Falk Matt Schrage The terminal is a necessary tool for any software engineer. In order to work quickly, developers have always customized their terminals to work for their specific application workflow. Fig is a reimagined terminal product that adds autocomplete and an entire app ecosystem to the existing terminal you are familiar with. In The post Fig Engineering with Brendan Falk and Matt Schrage appeared first on Software Engineering Daily.
4 May 2022 •
WP Engine is a domain specific cloud provider that hosts high performance WordPress infrastructure. This website, Software Engineering Daily, runs on WP Engine. Scaling a domain specific cloud provider for WordPress includes complexities at the level of the database, application, load balancer, and other areas. Brandon DuRutte from WP Engine joins the show to talk The post Scaling WordPress with Brandon DuRette appeared first on Software Engineering Daily.
3 May 2022 •
Data loss can occur when large data sources such as Slack or Google Drive get leaked. In order to detect and avoid leaks, a data asset graph can be built to understand the risks of a company environment. Polymer is a data loss prevention product that helps companies avoid problematic data leaks. Yasir Ali is The post Data Loss Prevention with Yasir Ali appeared first on Software Engineering Daily.
29 April 2022 •
The Presto/Trino project makes distributed querying easier across a variety of data sources. As the need for machine learning and other high volume data applications has increased, the need for support, tooling, and cloud infrastructure for Presto/Trino has increased with it. Justin Borgman is the CEO of Starburst, and joins the show to talk about The post Starburst Infrastructure with Justin Borgman appeared first on Software Engineering Daily.
28 April 2022 •
Data integration infrastructure is not easy to build. Moving large amounts of data from one place to another has historically required developers to build ad hoc integration points to move data between SaaS services, data lakes, and data warehouses. Today, there are dedicated systems and services for moving these large batches of data. Airbyte builds The post Airbyte Engineering with Michel Tricot appeared first on Software Engineering Daily.
27 April 2022 •
Modern organizations eventually face data governance challenges. Keeping track of where data came from, what systems update it, in what ways updates can be made are just some of the issues to be tackled. Large organizations face additional challenges like training, onboarding, and capturing the institutional knowledge that leaves with the departure of key team The post Select Star with Shinji Kim appeared first on Software Engineering Daily.
25 April 2022 •
23 April 2022 •
Loft is a platform for Kubernetes self-service and multi-tenancy. Loft allows you to control Kubernetes clusters with added multi-tenancy and self-service capabilities to get more value out of Kubernetes beyond simply cluster management. It allows for cost optimization, more efficient provisioning, and other features. Lukas Gentele joins the show to talk about Kubernetes multi-tenancy and The post Loft Kubernetes Namespaces with Lukas Gentele appeared first on Software Engineering Daily.
21 April 2022 •
Ben Sigelman Alex Kehlenbeck Observability consists of metrics, logs, and traces. Lightstep is a company that builds distributed tracing infrastructure, which requires them to store and serve high volumes of trace data. There are numerous architecture challenges that come with managing this data. Ben Sigelman and Alex Kehlenbeck join the show to discuss the implementation The post Distributed Tracing Infrastructure with Ben Sigelman and Alex Kehlenbeck appeared first on Software Engineering Daily.
19 April 2022 •
The solution many turn to for capturing their streaming data is InfluxDB. In this episode, I interview Brian Gilmore, Director of Product Management at InfluxData, about how real time applications achieve success built on top of InfluxDB. When most people hear the phrase Internet of Things, it typically evokes an image of connected devices we The post Time Series IoT on InfluxDB with Brian Gilmore appeared first on Software Engineering Daily.
14 April 2022 •
As companies move to Spark and a Lakehouse architecture, they are realizing that the data tools are lagging way behind. You need to be a programmer to effectively use Spark and Airflow. There are some low-code ETL tools, but is that enough? Companies want to treat their data pipelines like mission-critical apps. They want DevOps The post Lakehouse Data Stack with Raj Bains appeared first on Software Engineering Daily.
12 April 2022 •
Terminals are a gateway to building and running software, but they have not been reimagined since their initial creation. Warp is a new kind of terminal with visual aids, autocomplete, documentation, customization, and other features. It includes GPU acceleration, its own UI framework, and other engineering systems to increase speed and performance. Zach Lloyd joins The post Warp Terminal with Zach Lloyd appeared first on Software Engineering Daily.
7 April 2022 •
Lior Gavish James Densmore Data infrastructure is a fast-moving sector of the software market. As the volume of data has increased, so too has the quality of tooling to support data management and data engineering. In today’s show, we have a guest from a data intensive company as well as a company that builds a The post Data Engineering Trends with Lior Gavish and James Densmore appeared first on Software Engineering Daily.
5 April 2022 •
Gabe Greenberg Michelle Bakels Software engineering can be a surprisingly grueling career. It is both physically and mentally demanding to sit in front of a screen for hours on end producing code. During the pandemic, the strains of being an engineer became even more acute, as we were secluded in our homes, in some cases The post Developer Health with Gabe Greenberg and Michelle Bakels appeared first on Software Engineering Daily.
2 April 2022 •
Crypto companies have cemented themselves as a company category that is not going away. Bitski is a company that makes web3 tooling and infrastructure, including a wallet and a collection of tools for creating and selling NFTs. Patrick Tescher joins the show to talk through the engineering behind crypto infrastructure and APIs. Sponsorship inquiries: firstname.lastname@example.org The post Bitski: Web3 Infrastructure with Patrick Tescher appeared first on Software Engineering Daily.
1 April 2022 •
Running a database company requires expertise in both technical and managerial skills. There are deeply technical engineering questions around query paths, scalability, and distributed systems. And there are complex managerial questions around developer productivity and task allocation. Sam Lambert is the CEO of PlanetScale, which is building modern relational database infrastructure. Before PlanetScale he spent The post PlanetScale Management with Sam Lambert appeared first on Software Engineering Daily.
31 March 2022 •
Alexis Richardson Mohamed Ahmed GitOps is a deployment and infrastructure management strategy based around continuous delivery and Kubernetes, with Git at the center of deployment workflows. Policy management can be used to define permissions and rules around who can deploy and what constitutes safe deployment. There is a synergy between GitOps tooling and policy management The post GitOps and Policy Management with Alexis Richardson and Mohamed Ahmed appeared first on Software Engineering Daily.
30 March 2022 •
A company’s response to an incident defines how that company responds to an adverse, unexpected scenario. Kintaba automates the incident management process so teams can quickly report, respond, resolve, and reflect on major incidents collaboratively. Kintaba’s incident response platforms gives you instant access to incident management processes based on the practices of companies like Facebook, The post Kintaba Incident Response with John Egan appeared first on Software Engineering Daily.
29 March 2022 •
SingleStore is a multi-use, multi-model database designed for transactional and analytic workloads, as well as search and other domain specific applications. SingleStore is the evolution of the database company MemSQL, which sought to bring fast, in-memory SQL database technology to market. Jordan Tigani is Chief Product Officer of SingleStore and joins the show to talk The post SingleStore with Jordan Tigani appeared first on Software Engineering Daily.
29 March 2022 •
Compute resources continue to trend towards being cheaper, easier to use, and faster. Despite these positives, more compute demands more energy and therefore an increasing carbon footprint. With many companies committing to controlling their net carbon emissions, tools are required for engineers to not only measure their cloud infrastructure, but to make informed choices about The post Cloud Carbon Footprint with Steren Giannini appeared first on Software Engineering Daily.
25 March 2022 •
Vantage is a system for optimizing cloud costs. It provides tools and interfaces for developers to analyze how they are spending on AWS resources, and has recently expanded into GCP as well. Vantage users gain an easy interface into their costs that would otherwise be hard to analyze via the raw AWS console. Ben Schaechter The post Vantage Engineering with Ben Schaechter appeared first on Software Engineering Daily.
24 March 2022 •
At Lyft, Ketan Umare worked on Flyte, an orchestration system for machine learning. Flyte provides reliability and APIs for machine learning workflows, and is used at companies outside of Lyft such as Spotify. Since leaving Lyft, Ketan founded Union.ai, a company focused on productionizing Flyte as a service. He joins the show to talk about The post Union.ai with Ketan Umare appeared first on Software Engineering Daily.
23 March 2022 •
DuckDB is a relational database management system with no external dependencies, with a simple system for deployment and integration into build processes. It enables complex queries in SQL with a large function library, and provides transactional guarantees through multi-version concurrency control. Hannes Mühleisen works on DuckDB and joins the show to talk about query engines The post DuckDB with Hannes Muleisen appeared first on Software Engineering Daily.
19 March 2022 •
Substack is a platform for subscription based content, covering newsletters, podcasts, and soon video. The design and speed of the platform are notable–with a minimalistic format that simply gets creators paid for its work, Substack has gained massive popularity. Chris Best is the CEO of Substack and joins the show to talk about the platform, The post Substack with Chris Best appeared first on Software Engineering Daily.
17 March 2022 •
Customer data pipelines power the backend of many successful web platforms. In a customer data pipeline, data is collected from sources such as mobile apps and cloud SaaS tools, transformed and munged using data engineering, stored in data warehouses, and piped to analytics, advertising platforms, and data infrastructure. RudderStack is an open source customer data The post RudderStack Engineering with Soumaydeb Mitra appeared first on Software Engineering Daily.
16 March 2022 •
Managing Kubernetes nodes leads to operational complexity, security issues, and nodes that are perhaps more expensive to run than necessary. Deferring the node management to an underlying platform abstracts away these problems and can improve operations. Madhuri Yechuri runs Elotl, a nodeless Kubernetes platform. She joins the show to talk about the architecture and purpose The post Nodeless Kubernetes with Madhuri Yechuri appeared first on Software Engineering Daily.
16 March 2022 •
15 March 2022 •
The software supply chain consists of packages, imports, dependencies, containers, and APIs. These different components each have unique security risks. To ensure the security of their software supply chain, many developers use tools to analyze and scan their infrastructure for vulnerabilities. Barak Schoster works at Bridgecrew, a DevSecOps cloud security platform. He joins the show The post Software Supply Chain with Barak Schoster appeared first on Software Engineering Daily.
11 March 2022 •
The data lake architecture has become broadly adopted in a relatively short period of time. In a nutshell, that means data in it’s raw format stored in cloud object storage. Modern software and data engineers have no shortage of options for accessing their data lake, but that list shrinks quickly if you care about features The post Apache Hudi with Vinoth Chandar appeared first on Software Engineering Daily.
9 March 2022 •
National Instruments develops software and hardware for engineering in a wide variety of domains, from aerospace to government technology to application testing. The interface between hardware and software presents a variety of difficult engineering challenges. Luke Schreier is a Senior Vice President at National Instruments and joins the show to discuss the engineering and management The post National Instruments with Luke Schreier appeared first on Software Engineering Daily.
8 March 2022 •
Automating video search requires a data pipeline that extracts metadata from videos and allows users to annotate the video with information that correlates with that metadata. The video needs to be segmented into intervals that define the search space, and the search space needs to be queryable by a user. Sieve Data is an automated The post Video Search with Mokshith Voodarla appeared first on Software Engineering Daily.
4 March 2022 •
Increasingly, technology groups need to be strategic about the cloud services they adopt to ensure their vendor’s pricing is both fair and measured exactly right for the application’s unique access pattern. Definitely answering questions like these can take a significant amount of time and energy from your most valuable engineers. Taloflow is a company trying The post Taloflow with LV Jadavji appeared first on Software Engineering Daily.
3 March 2022 •
A data catalog provides an index into the data sets and schemas of a company. Data teams are growing in size, and more companies than ever have a data team, so the market for data catalog is larger than ever. Mark is the CEO of Stemma and the co-creator of Amundsen, a data catalog that came out of The post Data Catalog in Practice with Mark Grover appeared first on Software Engineering Daily.
25 February 2022 •
Splunk is a monitoring and logging platform that has evolved over its 18 years of existence. In its modern focus on observability it is focused on open source and AIOps. Observability has evolved with the growth of Kubernetes, and Splunk’s work around OpenTelemetry has kept parity with the open source community of Kubernetes. Spiros Xanthos The post Splunk Platform with Spiros Xanthos appeared first on Software Engineering Daily.
23 February 2022 •
Francesco Cesarini founded Erlang Solutions in 1999 with a mission to help companies adopt Erlang. In this interview, I speak with Francesco and Gabor Olah from Erlang Solutions. We discuss the Erlang language, it’s ecosystem, and features like concurrency, resilience, and scalability that motivate adoption. We use Java and the Java Virtual Machine as a The post Erlang Deep Dive with the Erlang Solutions team appeared first on Software Engineering Daily.
21 February 2022 •
Serious software projects require several environments. Your production environment is obviously mission critical. A staging environment is also necessary to perform validation and regression testing before taking the risk of pushing an update to production. Best practices and approaches for managing these and other environments vary from organization to organization. In some sense, different software The post The Staging Dichotomy with Senthil Padmanabhan appeared first on Software Engineering Daily.
19 February 2022 •
Barry McCardel Co-Founder and CEO at Hex Caitlin Colgrove Co-Founder and CTO at Hex In contrast to other IDEs, the notebook interface offers software developers a unique environment idealized for data professionals. Despite the growth in popularity, a surprising learning curve still exists for setup and configuration. A siloed notebook offers no native collaboration tools. The post Hex Collaborative Data Workspace with Barry McCardel and Caitlin Colgrove appeared first on Software Engineering Daily.
18 February 2022 •
When writing code, test driven development is a common accepted methodology to ensure the development of high quality software. Your organization’s data, on the other hand, is an entirely different challenge. Data can be missing due to human error, a failure with a 3rd party provider, a botched release, or dozens of other issues. When The post Data Quality Using Anomalo with Jeremy Stanley appeared first on Software Engineering Daily.
17 February 2022 •
When you visit a web page, the creator’s intent is to present you a seamless experience that fills your browser window. That web page or web application is generally divided up in some meaningful way across navigation elements, content, ads, header, footer, and other components. Those components may represent the work of independent teams. Typically The post Micro-Frontends with Luca Mezzalira appeared first on Software Engineering Daily.
15 February 2022 •
Lee Atchison spent seven years at Amazon working in retail, software distribution, and Amazon Web Services. He then moved to New Relic, where he has spent four years scaling the company’s internal architecture. From his decade of experience at fast-growing web technology companies, Lee has written the book Architecting for Scale, from O’Reilly. As an The post Architecting for Scale with Lee Atchison appeared first on Software Engineering Daily.
13 February 2022 •
The creator economy has seen rapid growth, thanks largely to software solutions like Patreon that are enabling creators. As the creator economy grows, providers must be prepared for scalability issues and the challenges of maintaining and growing a software infrastructure and the team that evolves it. In this episode, I interview Utkarash Srivastava, SVP Engineering The post Patreon Engineering with Utkarsh Srivastava appeared first on Software Engineering Daily.
9 February 2022 •
Snyk is a platform for security that started with open source scanning and has expanded into container security, infrastructure as code, and other products. Snyk is a simple product to use, but has hidden complexities that build large data structures to manage and scan code dynamically. In a previous episode we discussed the core Snyk The post Snyk Engineering with Guy Podjarny appeared first on Software Engineering Daily.
7 February 2022 •
Linkerd is a service mesh that runs efficiently with a low memory footprint. We have covered the details of Linkerd in previous episodes. Buoyant is the company that sells Linkerd as a service, and today’s show focuses on the engineering details of the company, and how Linkerd is architected in 2022. William Morgan is the The post Buoyant Cloud with William Morgan appeared first on Software Engineering Daily.
4 February 2022 •
Online poker was deemed illegal in the United States ten years ago. Since then, poker has decreased in popularity, then found new invigoration thanks to live streaming and a large volume of bored gamers looking for something to do during the pandemic. Poker is a strategy game that can be played even without the financial The post Clubs Poker with Taylor Crane appeared first on Software Engineering Daily.
1 February 2022 •
Database product companies typically have a few phases. First, the company will develop a technology with some kind of innovation such as speed, scalability, or durability. The company will offer support contracts around that technology for a period of time, before eventually building a managed, hosted offering. PlanetScale is a database company built around the The post Scaling PlanetScale with Sugu Sougoumarane appeared first on Software Engineering Daily.
31 January 2022 •
Couchbase is a distributed NoSQL cloud database. Since its creation, Couchbase has expanded into edge computing, application services, and most recently a database-as-a-service called Capella. Couchbase started as an in-memory cache and needed to be rearchitected to be a persistent storage system. In this episode, I interview Ravi Mayuram, SVP Products and Engineering at Couchbase The post Couchbase Architecture with Ravi Mayuram appeared first on Software Engineering Daily.
28 January 2022 •
Rackspace is a multi cloud solutions provider that has evolved beyond its cloud computing origins into a diverse set of services and support offerings. Customers work with Rackspace to adopt cloud application deployments, modern data analytics, and all the other opportunities offered by cloud computing. Much of this occurs through partnerships where Rackspace provides teams The post Rackspace with Jeff DeVerter appeared first on Software Engineering Daily.
27 January 2022 •
Historically, search engines made money by showing sponsored ads alongside organic results. As the idiom goes, if you’re not paying for something, you are the product. Neeva is a new take on search engines. When you search at neeva.com, you get the type of result you’d expect from a search engine minus any advertising. In The post Ad-free Search on Neeva with Darin Fisher appeared first on Software Engineering Daily.
25 January 2022 •
Tabnine is an AI assistant that helps software engineers write more efficient code. It’s been trained on a large corpus of source code or can be trained based on your specific codebase. Either way, the resulting model offers predictive completion of code that can make programmers more productive. In this episode, I interview Eran Yahav, The post Tabnine with Eran Yahav appeared first on Software Engineering Daily.
21 January 2022 •
Protecting your customers begins with best practices for securely capturing, storing, and protecting the data you collect for or about them. When an organization has a large enough dataset, needs typically arise for doing analytical workloads or training machine learning models on this data. If you use random or mock data to generate a report The post Privacy Engineering with Alex Watson appeared first on Software Engineering Daily.
20 January 2022 •
When businesses share a common need such as payroll, commercial offerings can compete for market share with software solutions that easily adapt to a variety of businesses. Not all tasks can be easily commoditized or standardized. Take content moderation as an example. Every site that accepts user generated content is likely to have unique and The post Flowdash with Nick Gervasi appeared first on Software Engineering Daily.
19 January 2022 •