Articles & Tutorials

Get started with me & Kestra.io

Guess who got a new job and is going to take you along the onboarding and learning journey? ME! 🤗 I decided to go back to the core of my DevRel roots and join Kestra.io as a developer advocate to help expand into the US market. I'm pretty excited to get back to learning and doing in public, and of course seeing all your wonderful faces in person at events. I've really missed the direct connection to the developer community. It's good to be back. What is Kestra? Kestra is a declarative workflow orchestration platform that really leans into the everything-as-code construct, but also has a pretty good experience for the config only no-coders out there too. Since I enjoy walking the line between coding and no-coding experiences out of pure laziness, the editor experience really sold it for me. I can see the YAML (code), no-code (that immediately modifies the YAML), and an editable topology all in the same editing experience in side-by-side panels. The YAML is readable, and the diagram gives me a great visual of what I'm building, and I can edit wherever I want. This really lights up all the areas in my brain in ways other platforms just haven't done. Leaning too hard into no-code and config only experiences made me frustrated that I couldn't just write code, while sometimes I'd love to have someone else wrap the values I need for my solution in the boilerplate necessary to just make them work. Callback to the next Hannah Montana gif because I do, in fact, like the flexibility that comes with the best of both worlds. Using and evaluating Kestra Kestra has two editions - OSS and Enterprise - although it's three if you count Cloud. For the purposes of this blog, Cloud is just a different administered experience, a different flavor of Enterprise if you will. If you are looking for an OSS project to contribute to in the new year, we got that too! But I digress. Kestra OSS will be great for single user use cases, but also more than enough for doing a general evaluation of the flow building experience. Kestra Enterprise does unlock some pretty cool enterprise-y features, but digging into building your first flow is where things will click. At least that's where they did for me. I recommend following the quickstart and spinning up Kestra OSS with Docker compose. Just make sure you have Docker setup and, unless you are old like me, you probably already know Docker compose ships with Docker. What a time to be alive! Pop this into your terminal, then head to http://localhost:8080: docker run --pull=always --rm -it -p 8080:8080 --user=root -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp kestra/kestra:latest server local Pay attention to the note in the docs about whether you need a persistent database backend or not. For my first few flows and executions this wasn't necessary and, to be honest, I just left it running in the background because I wasn't doing anything else that was compute heavy or intensive. Your mileage may vary! I then recommend heading straight to the Blueprints after running a tutorial or two. Once you execute a few flows to get some data in your dashboards you can dig in to building your own flow with the tons of available community plugins. I'm still working on a flow that prompts me to wrap up my day or week by pulling together what I did in different places so I can accurately summarize what I actually did. Maybe remind me to do a blog on this later. Keen eyes may notice my WIP in the flow editor screenshot. 👀 But wait, what's an orchestrator even for? I have this hypothesis that "orchestration" doesn't resonate with everyone in tech. At least not yet. I think many of us have a general understanding that we have many disparate systems with data everywhere and if we could just use all of that together, we could maybe solve some really cool and interesting problems. That's where the orchestration piece comes in - getting these different systems to talk to each other to accomplish a unified goal or a task. The folks in DevOps seem to get it, along with the folks in AI, probably because they are already working with so many different pieces and pulling things together, whether it's 5 different platforms and projects to standup a cluster or merging datasets from several different sources to provide the best answer. I'm sure I'll have another blog on this topic in the future, but I'm interested to hear what your definition of an orchestrator or orchestration is, when you learned about it, and how you use it, or maybe even how you know when to use an orchestrator. Leave me a comment and let's talk! Join me as I dig in more and let me know what you are building. Like I said, it's good to be back. 💜

dev.to

'Big Brother' star Mickey Lee dies at 35 after cardiac arrests from flu complications

"Big Brother" alum Mickey Lee died on Thursday after suffering multiple heart attacks following flu complications, according to her family.

nbcnews.com

Forecasting Appointment No-Shows and Improving Healthcare Access: A Machine Learning Framework

The Business Impact of No-Shows Healthcare appointment no-shows create a cascading effect on system performance. When patients miss appointments without cancellation, medical facilities lose the opportunity to serve other patients who need care. The consequences include increased healthcare costs, wasted clinical resources, and reduced provider productivity. In rural healthcare settings, where access is already limited, the impact becomes even more pronounced.​ Recent studies have demonstrated that AI-based appointment systems can increase patient attendance rates by 10% per month and improve hospital capacity utilization by 6%. These improvements translate directly to enhanced service quality and reduced operational costs.​ Key Predictive Features Research across multiple healthcare systems has identified consistent risk factors for appointment no-shows:​ Previous no-show history: Patients with no-show records in the last three months have 4.75 times higher odds of missing their next appointment Appointment rescheduling: Rescheduled appointments show significantly higher no-show rates Lead time: Longer intervals between scheduling and appointment dates increase no-show probability Payment method: Self-pay patients demonstrate higher no-show rates compared to insured patients Appointment confirmation status: Patients who don't confirm via automated systems are at elevated risk Demographics: Age, gender, and geographic location contribute to prediction accuracy Studies show that patients with multiple previous no-shows can have no-show rates as high as 79%, compared to just 2.34% for patients with clean attendance records.​ Building a Prediction Model *Data Collection and Preparation * Start by gathering historical appointment data from your electronic health records (EHR) system. A robust model requires a substantial dataset—one study used over 1.2 million appointments from 263,464 patients. Essential features include:​ Patient demographics (age, gender, address) Appointment characteristics (date, time, specialty, provider) Insurance and payment information Historical attendance patterns Lead time and rescheduling indicators *Model Selection * Multiple machine learning approaches have proven effective for no-show prediction:​ Logistic Regression: Provides interpretable odds ratios and probability estimates, making it ideal for understanding risk factors. This approach allows healthcare administrators to estimate the change in risk associated with specific patient characteristics.​ Decision Trees: Offer intuitive rule-based predictions that clinical staff can easily understand and apply.​ Advanced Algorithms: JRip and Hoeffding tree algorithms have achieved strong predictive performance in hospital settings.​ Recent research demonstrates that machine learning models can achieve accuracy scores of 0.85 for predicting no-shows and 0.92 for late cancellations on a 0-1 scale.​ *Implementation Approach * python # Conceptual framework for no-show prediction import pandas as pd from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import classification_report, roc_auc_score # Load and prepare appointment data appointments_df = load_appointment_data() # Feature engineering features = [ 'days_until_appointment', 'previous_noshow_count_3months', 'appointment_rescheduled', 'self_pay_flag', 'appointment_confirmed', 'patient_age', 'appointment_hour' ] X = appointments_df[features] y = appointments_df['no_show'] # Train-test split X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, stratify=y ) # Train model model = LogisticRegression() model.fit(X_train, y_train) # Evaluate performance predictions = model.predict_proba(X_test)[:, 1] print(f"AUC-ROC: {roc_auc_score(y_test, predictions)}") Operationalizing Predictions *Risk Stratification * Develop a tiered risk classification system that categorizes appointments by no-show probability. For example:​ Category 0: 0-10% no-show risk (2-3% actual no-show rate) Category 1-2: 10-30% risk Category 3-4: 30-60% risk Category 5: 60%+ risk (up to 79% actual no-show rate) *Targeted Interventions * Deploy different intervention strategies based on risk level:​ High-risk patients: Automated callback systems to confirm attendance, SMS reminders, flexibility to reschedule Medium-risk patients: Multiple reminder touchpoints via text and phone Low-risk patients: Standard single reminder One healthcare system successfully implemented AI-driven callbacks using VoiceXML and CCXML technologies to confirm high-risk appointments, creating detailed risk profiles based on patient history and demographics.​ *Intelligent Overbooking * Use prediction models to optimize scheduling through strategic overbooking. Research suggests that one overbook should be scheduled for every six at-risk appointments, balancing the risk of no-shows against potential overbooking. This data-driven approach increases treatment availability while maintaining service quality.​ *Measuring Success * Track these key performance indicators to evaluate your no-show reduction program: Overall no-show rate reduction Capacity utilization improvement Patient satisfaction scores Provider productivity metrics Cost savings from reduced waste One organization successfully reduced no-show rates from 49% to 18% and maintained rates below 25% for two years through improved communication and appointment flexibility.​ Ethical Considerations When implementing predictive models for healthcare, consider: Bias mitigation: Ensure models don't discriminate against vulnerable populations Transparency: Communicate with patients about how predictions inform scheduling Privacy: Protect patient data according to HIPAA and other regulations Fairness: Use predictions to improve access, not restrict it for high-risk groups .​

dev.to

Trump and Zelenskyy to meet at Mar-a-Lago

Trump and Zelenskyy to meet at Mar-a-Lago

nbcnews.com

There's a spell for that: Why 2025 was the year of the Etsy witch

While witchcraft is an ancient practice and Etsy witches have existed online for years, interest in buying quick spells online surged in 2025 after several influencers praised their results.

nbcnews.com

CODIMAP - Fantasy map tool

Hi! 👋 I’d like to share a project I recently created out of frustration with the lack of simple and accessible world-building tools. My application is called CodiMap Check Codimap here In CodiMap, maps are created in a fantasy style. The tool was built specifically to help users pour their ideas onto a “digital canvas” and create their dream worlds without the need to learn complicated graphic design software. Within this visual style, the application offers dedicated features that make building realms easier: use of various terrain types, creation of political maps, adding cities and custom text. All of these elements allow users to prepare high-resolution maps, which can then be exported to PNG or JSON formats. I developed the program using JavaScript and the Electron framework. I’m continuously developing the project, regularly adding new features and improvements to ensure the application serves the community as well as possible. Currently, a Windows version is available and can be downloaded from itch.io, or run directly in the browser.

dev.to

$1.8 billion Powerball ticket sold in Arkansas

$1.8 billion Powerball ticket sold in Arkansas

nbcnews.com

WebClient ColdStart 문제

컬리, CJ, 쿠팡 등 물류 인프라를 보유하고 있는 회사들은 3PL이라는 서비스를 제공합니다. 3PL이란 물류 인프라를 갖춘 회사가 그렇지 못한 판매처로부터 배송 업무를 위탁받아 제공하는 서비스를 말합니다. 판매처는 배송이 필요한 주문 목록을 3PL 시스템에 등록하게 되는데, 이 과정에서 입력된 주문이 유효한 주문인지 확인하기 위해서 여러 시스템과 소통하게 됩니다. 외부 API 호출을 위한 도구로는 WebClient를 사용하고 있었는데요. 아래 지표에서 보듯이, 외부 API 호출까지의 지연 시간이 원인을 알 수 없이 길어지는 현상이 '간헐적'으로 발견되었습니다. 이번 포스트에서는 해당 현상의 원인을 파악하며 알게 된 WebClient의 내부 동작 원리와 Reactor Netty의 아키텍처, 그리고 해결책을 공유하고자 합니다. 원인 파악을 위한 가정 지연이 발생한다는 것은 요청이 어딘가에서 즉시 처리되지 못하고 대기하고 있었을 가능성이 높다는 의미입니다. 이전 사진에서 빨간 박스로 표시된 메서드는 객체 생성과 WebClient 호출만을 담당하고 있었습니다. WebClient 내부의 어느 처리 단계에서 병목이 발생할 수 있는지 명확히 식별하려면 아키텍처에 대한 이해가 선행되어야 했습니다. 이를 위해 WebClient의 요청 처리 방식과 Reactor Netty의 아키텍처를 분석했습니다. WebClient 실행 메커니즘 먼저 문제가 발생한 코드의 구조를 살펴보겠습니다. OrderRegistrationService.java private final static int CONCURRENCY_CALL = 10; List<RefineResult> results = Flux.fromIterable(registerDtos) .flatMap(dto -> Mono.defer(() -> omsService.refineAddress(dto.getPrimaryAddress(), dto.getSecondaryAddress()) .defaultIfEmpty(ApiResponse.failure("NO_RESPONSE", false)) ).map(resp -> new RefineResult(dto, resp)), CONCURRENCY_CALL) .collectList() .block(); OmsService.java @Override public Mono<ApiResponse<RefineAddressDto>> refineAddress(String primaryAddress, String secondaryAddress) { RefineAddressInput request = RefineAddressInput.builder() .primaryAddress(primaryAddress) .secondaryAddress(secondaryAddress) .build(); return omsClient.post() .uri("/refine-address") .bodyValue(request) .retrieve() .bodyToMono(RefineAddressOutput.class) .timeout(Duration.ofSeconds(5)) .retryWhen(RetryPolicy.fixedDelay(1, Duration.ofMillis(100), "[OMS 주소정제] 재시도 요청: " + request)) .map(output -> output.isSuccess() ? ApiResponse.success(RefineAddressDto.from(output)) : ApiResponse.<RefineAddressDto>failure("응답 결과에 데이터가 없음", false)) .onErrorResume(ex -> ExternalErrorHandler.handleError(ex, extractOmsErrorMessage(ex), "OMS 주소정제")); } Cold Sequence와 구독 시점 위 코드에서 실제 HTTP 요청이 언제 발생하는지 이해하려면 WebClient의 Cold Sequence 특성을 먼저 이해해야 합니다. WebClient의 리액티브 체인은 Cold Sequence로 동작합니다. 구독이 발생하기 전까지는 파이프라인만 정의될 뿐, 실제 실행은 일어나지 않습니다. HTTP 요청 발송 시점은 subscribe()가 호출되는 순간이며, 코드상의 block()이 내부적으로 이를 트리거합니다. List<RefineResult> results = Flux.fromIterable(registerDtos) .flatMap(dto -> Mono.defer(() -> ...)) .collectList() .block(); // ← 구독 시작점 block()의 구독 신호는 역방향(upstream)으로 전파됩니다 block() → collectList() → flatMap() → Mono.defer() → WebClient 체인 flatMap(Function, int concurrency)은 인자로 전달된 concurrency 수 만큼의 Mono를 동시에 구독합니다. Mono.defer()는 각 구독 시점마다 내부 람다를 실행하여 새로운 Mono를 생성하므로, 각 DTO마다 독립적인 HTTP 요청 파이프라인이 생성됩니다. // 구독될 때마다 새로운 WebClient 체인 생성 Mono.defer(() -> omsService.refineAddress(...)) TaskQueue로의 전달 omsService.refineAddress(...)가 반환하는 Mono가 구독되면 요청 설정을 빌드하고, .retrieve() 이후 체인이 구독되면서 쓰기 요청이 TaskQueue에 저장됩니다. omsClient.post() .uri("/refine-address") .bodyValue(request) .retrieve() .bodyToMono(RefineAddressOutput.class) POST 요청이 NioEventLoop의 TaskQueue에 저장되면, WebClient를 호출한 스레드의 역할은 여기서 끝납니다. 이후 작업은 EventLoop 스레드가 담당합니다. Netty EventLoop 스레드의 동작 원리 WebClient의 HTTP 요청이 TaskQueue에 저장되는 이유는 Netty의 이벤트 루프 기반 비동기 처리 모델 때문입니다. 이 모델을 이해하려면 먼저 네트워크 통신의 기본 개념을 짚고 넘어가야 합니다. User Space와 Kernel Space 서로 다른 머신의 애플리케이션이 통신하려면 시스템 콜로 유저 모드와 커널 모드를 오가며 커널 내 소켓 버퍼에 데이터를 읽거나 써야 합니다. 소켓 버퍼에 데이터를 어떻게 읽고 쓰느냐에 따라 Blocking I/O와 Non-blocking I/O로 나뉩니다. 둘의 차이는 스레드가 시스템 콜 후 응답을 기다리는지 여부입니다. Blocking I/O: 데이터가 준비될 때까지 스레드가 대기 Non-blocking I/O: 데이터가 없으면 즉시 반환, 스레드는 다른 작업 수행 가능 효율적인 Non-blocking I/O를 구현하려면 특정 이벤트를 등록해 놓고 해당 이벤트가 발생했을 때만 처리하는 방식이 필요합니다. 이렇게 하면 하나의 스레드로 여러 채널을 관리할 수 있습니다. Multiplexing I/O와 Selector 이벤트 기반 소켓 통신에서는 하나의 Selector가 여러 소켓 채널의 변화를 감지하며 이벤트가 발생했을 때만 처리합니다. 이를 Multiplexing I/O라고 합니다. Linux에서 이 Multiplexing I/O는 epoll 시스템 콜로 구현됩니다. Java NIO의 Selector는 내부적으로 이 epoll을 사용합니다. Selector.select()의 실제 동작 OS 커널이 능동적으로 I/O 이벤트를 Selector에 알려주는 것처럼 보이지만, 실제로는 그렇지 않습니다. Selector.select()가 호출되면 유저 모드에서 커널 모드로 전환되고, 내부적으로 epoll_wait() 시스템 콜이 호출되면서 호출 스레드는 커널에서 블로킹 상태로 대기합니다. epoll_wait을 호출하면, OS 커널은 이전에 epoll_ctl로 등록된 파일 디스크립터(소켓)들을 모니터링하다가, 네트워크 카드에 데이터가 도착하거나 소켓 버퍼에 쓰기가 가능해지는 등의 I/O 이벤트가 발생하면 이를 감지합니다. I/O가 발생한 소켓은 커널 내 Ready Queue에 추가되고, epoll_wait()이 반환되어 대기 중이던 스레드가 깨어납니다. 즉, User Space가 커널에 요청하고 시스템 콜로 응답받는 pull 구조입니다. select() 자체는 블로킹 호출이지만, 하나의 스레드가 여러 소켓을 감시하고 이벤트가 발생한 소켓들만 골라서 처리합니다. 따라서 각 소켓 입장에서는 전용 스레드 없이도 비동기적으로 처리되는 것과 같은 효과를 얻게 됩니다. NioEventLoop의 구조 이제 Netty의 EventLoop가 Selector를 어떻게 활용하는지 살펴보겠습니다. EventLoop의 구현체인 NioEventLoop는 1 Thread + 1 Selector + 1 TaskQueue로 구성됩니다. EventLoop 스레드는 기본적으로 CPU 코어 수만큼 생성됩니다. Math.max(Runtime.getRuntime().availableProcessors(), 4) 각 EventLoop 스레드는 전용 NioEventLoop 인스턴스를 실행하며, 단일 스레드가 무한 루프를 돌면서 두 가지 작업을 수행합니다. I/O 이벤트 처리 (네트워크 읽기/쓰기) TaskQueue의 작업 처리 (사용자가 등록한 Runnable) // 개념적인 코드 while (true) { // 1. 네트워크에서 뭔가 일어났는지 확인 네트워크_이벤트_확인(); // 2. 일어난 일들 처리 이벤트들_처리(); // 3. 누가 시켜놓은 작업들 처리 작업큐에서_작업꺼내서_실행(); } 실제 Netty 코드를 보면 (Netty 4.2 기준) // SingleThreadIoEventLoop.java:153-164 protected void run() { do { runIo(); // ← 1+2: I/O 확인 및 처리 runAllTasks(maxTasksPerRun); // ← 3: 작업큐 처리 } while (!confirmShutdown()); } runIo()는 내부적으로 NioIoHandler.run()을 호출합니다. // NioIoHandler.java:420-485 public int run(IoExecutionContext runner) { // 1단계: select - I/O 이벤트 존재 여부 확인 select(runner, wakenUp.getAndSet(false)); // 2단계: 있으면 처리 return processSelectedKeys(); } 이제 각 단계를 자세히 살펴보겠습니다. 1. select() - I/O 이벤트 감지 EventLoop는 I/O 이벤트 처리와 TaskQueue에 쌓인 작업 처리, 두 가지 역할을 수행합니다. 이때 Selector.select()를 사용하여 처리할 I/O 이벤트가 있는지 확인합니다. select() 메서드는 TaskQueue에 작업이 존재하는지 여부에 따라 적절한 select 방식을 결정합니다. // NioIoHandler.java private void select(IoExecutionContext runner, boolean oldWakenUp) { Selector selector = this.selector; for (;;) { // 태스크가 있으면 즉시 확인하고 넘어감 if (!runner.canBlock() && wakenUp.compareAndSet(false, true)) { selector.selectNow(); // 작업 있으면 바로 확인 break; } // 태스크가 없으면 이벤트 올 때까지 대기 int selectedKeys = selector.select(timeoutMillis); } } @Override public boolean canBlock() { assert inEventLoop(); return !hasTasks() && !hasScheduledTasks(); } TaskQueue가 비었을 때 TaskQueue가 비어있으면 Netty는 select(timeout)을 호출하여 커널로 부터 I/O 이벤트 신호를 받거나 타임아웃이 될 때까지 블로킹 상태로 대기하여 CPU 사용을 줄입니다. 만약 대기 중 TaskQueue에 새 task가 들어오면, wakeup 메커니즘을 통해 select()의 블로킹을 깨워서 즉시 반환시키고, 루프를 돌며 TaskQueue를 처리할 수 있게 합니다. TaskQueue가 있을 때 TaskQueue에 작업이 있으면 selectNow()를 호출하여 I/O 이벤트가 있는지 빠르게 확인하고, 곧바로 테스크 실행으로 넘어가 작업 지연을 줄입니다. 만약 TaskQueue에 작업이 있는 상황에서 select(timeout)을 호출해 블로킹되면, EventLoop 스레드가 잠들어 테스크 처리가 지연되고 응답성이 떨어지게 됩니다. 반대로 selectNow()만 계속 수행하면 준비된 I/O 이벤트가 없어도 계속 확인하므로 불필요한 반복으로 busy-wait(CPU 낭비)이 발생할 수 있습니다. 즉, Netty의 select()는 상황에 따라 적절한 방식을 선택하여 CPU를 낭비하지 않고 효율적으로 I/O 이벤트를 대기합니다. select() 호출 이후의 내부 동작 앞서 Netty가 상황에 따라 select(timeout) 또는 selectNow()를 선택적으로 호출한다는 것을 살펴보았습니다. 이제 이 호출이 실제로 어떤 과정을 거쳐 커널까지 도달하고, 다시 돌아오는지 살펴보겠습니다. Selector.select()를 호출하면 JDK 내부의 SelectorImpl 클래스가 이를 처리합니다. // SelectorImpl.java @Override public final int select(long timeout) throws IOException { return lockAndDoSelect(null, (timeout == 0) ? -1 : timeout); } lockAndDoSelect()는 동기화를 수행한 뒤 Multiplexing I/O를 담당하는 doSelect()를 호출합니다. // SelectorImpl.java private int lockAndDoSelect(Consumer<SelectionKey> action, long timeout) throws IOException { synchronized (this) { ensureOpen(); if (inSelect) throw new IllegalStateException("select in progress"); inSelect = true; try { synchronized (publicSelectedKeys) { return doSelect(action, timeout); } } finally { inSelect = false; } } } 여기서 doSelect()는 추상 메서드입니다. 운영체제마다 효율적인 Multiplexing I/O 메커니즘이 다르기 때문에, JDK는 플랫폼별로 다른 구현체를 제공합니다. OS 구현 클래스 시스템 콜 Linux EPollSelectorImpl epoll_wait() macOS KQueueSelectorImpl kevent() Windows WindowsSelectorImpl IOCP 이 글에서는 서버 환경에서 가장 많이 사용되는 Linux의 epoll 기반 구현을 중심으로 살펴보겠습니다. (JDK 21 기준) EPollSelectorImpl 인스턴스는 언제 생성되는가? EPollSelectorImpl 인스턴스는 Selector.open() 호출 시점에 초기화됩니다. 애플리케이션에서 new NioEventLoopGroup(n) 호출 내부적으로 n개의 NioIoHandler 생성 각 NioIoHandler 생성자에서 provider.openSelector() 호출 Linux 환경에서는 EPollSelectorImpl 인스턴스 생성 EPollSelectorImpl 생성 시 다음과 같은 초기화가 이루어집니다. EPollSelectorImpl(SelectorProvider sp) throws IOException { super(sp); // 1. epoll 인스턴스 생성 (epoll_create 시스템 콜) this.epfd = EPoll.create(); // 2. epoll_wait 결과를 저장할 네이티브 메모리 할당 this.pollArrayAddress = EPoll.allocatePollArray(NUM_EPOLLEVENTS); // 3. wakeup용 EventFD 생성 this.eventfd = new EventFD(); IOUtil.configureBlocking(IOUtil.newFD(eventfd.efd()), false); // 4. EventFD를 epoll에 EPOLLIN으로 등록 EPoll.ctl(epfd, EPOLL_CTL_ADD, eventfd.efd(), EPOLLIN); } 즉, 하나의 EventLoop마다 하나의 epoll 인스턴스가 매핑됩니다. epoll의 세 가지 시스템 콜 epoll은 세 가지 시스템 콜을 제공합니다 epoll_create: epoll 인스턴스(채널 감시 저장소) 생성 epoll_ctl: 감시할 FD 추가/수정/삭제 epoll_wait: 이벤트(read/write)가 발생할 때까지 대기하고, 이벤트가 발생한 FD 목록을 반환 JDK의 EPoll.wait()는 JNI를 통해 커널의 epoll_wait() 시스템 콜을 직접 호출합니다. epoll_wait()는 미리 할당된 네이티브 메모리의 epoll_event 구조체 배열에 준비된 이벤트 정보를 채우고, 준비된 이벤트 개수를 반환합니다. 이 배열에는 각 FD와 발생한 이벤트 타입(EPOLLIN/EPOLLOUT/EPOLLERR 등)이 담겨 있습니다. EPollSelectorImpl.doSelect()에서 이 메서드들이 실제로 호출되는 흐름을 보면: // EpollSelectorImpl.java @Override protected int doSelect(Consumer<SelectionKey> action, long timeout) throws IOException { int to = (int) Math.min(timeout, Integer.MAX_VALUE); int numEntries; processUpdateQueue(); // epoll_ctl로 관심 이벤트 변경 반영 processDeregisterQueue(); try { begin(blocking); // epoll_wait 시스템 콜 호출 numEntries = EPoll.wait(epfd, pollArrayAddress, NUM_EPOLLEVENTS, to); } finally { end(blocking); } // 반환된 이벤트 처리 return processEvents(numEntries, action); } 2. processSelectedKeys() - I/O 이벤트 처리 EPoll.wait()가 이벤트 개수를 반환하면, EPollSelectorImpl.processEvents()가 해당 개수만큼 이벤트 배열을 순회하며 각 FD에 연결된 SelectionKey를 찾아 selectedKeys에 추가합니다. 이후 Netty의 NioIoHandler.processSelectedKeys()가 seletedKeys를 순회하며 각 채널의 이벤트를 처리합니다. // NioIoHandler.java private int processSelectedKeysOptimized() { int handled = 0; for (int i = 0; i < selectedKeys.size; ++i) { SelectionKey k = selectedKeys.keys[i]; selectedKeys.keys[i] = null; // GC를 위해 null 처리 processSelectedKey(k); // 각 이벤트 처리 ++handled; } return handled; } private void processSelectedKey(SelectionKey k) { final DefaultNioRegistration registration = (DefaultNioRegistration) k.attachment(); // 준비된 이벤트를 핸들러에 전달 // OP_READ → 데이터 수신 // OP_WRITE → 데이터 송신 // OP_CONNECT → 연결 완료 // OP_ACCEPT → 새 연결 요청 registration.handle(k.readyOps()); } registration.handle()은 내부적으로 AbstractNioChannel.AbstractNioUnsafe.handle()을 호출합니다. 이 메서드는 이벤트 타입에 따라 적절한 처리를 수행합니다. // AbstractNioChannel.java:420-450 @Override public void handle(IoRegistration registration, IoEvent event) { NioIoOps nioReadyOps = ((NioIoEvent) event).ops(); // 1. OP_CONNECT: 연결 완료 처리 (가장 먼저 처리) if (nioReadyOps.contains(NioIoOps.CONNECT)) { removeAndSubmit(NioIoOps.CONNECT); unsafe().finishConnect(); } // 2. OP_WRITE: 쓰기 가능 상태 - 대기 중인 버퍼 전송 if (nioReadyOps.contains(NioIoOps.WRITE)) { forceFlush(); } // 3. OP_READ / OP_ACCEPT: 데이터 수신 또는 새 연결 수락 if (nioReadyOps.contains(NioIoOps.READ_AND_ACCEPT) || nioReadyOps.equals(NioIoOps.NONE)) { read(); } } 3. runAllTasks() - Non-I/O Task 처리 I/O 이벤트 처리가 끝나면 runAllTasks()가 호출되어 TaskQueue에 쌓인 작업들을 처리합니다. WebClient의 HTTP 요청도 바로 이 단계에서 실제로 전송됩니다. // SingleThreadEventExecutor.java protected boolean runAllTasks(long timeoutNanos) { // 스케줄 큐에서 실행 가능한 태스크를 TaskQueue로 이동 fetchFromScheduledTaskQueue(); Runnable task = pollTask(); final long deadline = timeoutNanos > 0 ? getCurrentTimeNanos() + timeoutNanos : 0; long runTasks = 0; for (;;) { safeExecute(task); // 테스크 실행 runTasks ++; // Check timeout every 64 tasks because nanoTime() is relatively expensive. // XXX: Hard-coded value - will make it configurable if it is really a problem. if ((runTasks & 0x3F) == 0) { lastExecutionTime = getCurrentTimeNanos(); if (lastExecutionTime >= deadline) { break; } } task = pollTask(); if (task == null) { lastExecutionTime = getCurrentTimeNanos(); break; } } afterRunningAllTasks(); return true; } 전체 흐름 요약 지금까지 살펴본 내용을 하나의 다이어그램으로 정리하면 다음과 같습니다. 병목이 발생한 인스턴스의 vCPU 수는 2개였습니다. Netty의 EventLoop 스레드 수는 기본적으로 Math.max(availableProcessors(), 4)로 결정되므로, 이 환경에서는 EventLoop 스레드가 총 4개 존재합니다. 지금까지 살펴본 바와 같이 Netty의 EventLoop는 Multiplexing I/O 방식으로 동작하기 때문에 적은 수의 스레드로도 많은 동시 요청을 처리할 수 있습니다. epoll_wait()은 수천 개의 채널이 등록되어 있어도 실제로 I/O 이벤트가 발생한 채널만 반환하므로, 동시 요청 수가 EventLoop 스레드 수보다 많다고 해서 병목이 발생하지는 않습니다. 따라서 "동시 요청 10개 > EventLoop 스레드 4개"는 병목의 원인이 아닙니다. 또 다른 가설: Parallel Scheduler 스레드 경합 그렇다면 무엇이 문제였을까요? 문제가 발생한 코드를 다시 살펴보겠습니다. return omsClient.post() .uri("/refine-address") .bodyValue(request) .retrieve() .bodyToMono(RefineAddressOutput.class) .timeout(Duration.ofSeconds(5)) // ← 여기 .retryWhen(RetryPolicy.fixedDelay(1, Duration.ofMillis(100), "...")) // ← 여기 // ... timeout()과 retryWhen()의 fixedDelay()는 내부적으로 Schedulers.parallel()을 사용하여 타이머를 스케줄링합니다. 그리고 Parallel Scheduler의 스레드 수 역시 CPU 코어 수에 비례합니다. vCPU 2개 환경에서 동시 요청 수가 10개인 환경에서 할당된 Parallel Scheduler 스레드 수보다 많은 요청을 처리하면서 병목이 발생했을 가능성이 있습니다. 검증 실제로 CPU 코어 수 제한이 WebClient 동시 호출 성능에 미치는 영향을 측정하기 위해 JMH(Java Microbenchmark Harness)를 사용하여 벤치마크를 수행했습니다. 테스트 환경 공통 설정 동시 호출 수 (Concurrency): 10 총 요청 수 (totalRequests): 50 서버 응답 지연 (serverLatencyMs): 200m 측정 방식: 10회 반복 측정 후 평균값 산출 Case 1: CPU 코어 10개 (refineAddress_concurrency10_cpu10) JVM 옵션: -XX:ActiveProcessorCount=10 Available Processors: 10 Reactor Schedulers DefaultPoolSize: 10 Case 2: CPU 코어 2개 (refineAddress_concurrency10_cpu2) JVM 옵션: -XX:ActiveProcessorCount=2 Available Processors: 2 Reactor Schedulers DefaultPoolSize: 2 결과 지표 CPU 10 코어 CPU 2 코어 평균 처리 시간 1035.498 ms 1049.652 ms 안정성 (Stdev) 7.496 18.554 신뢰구간 폭 22.667 ms 56.103 ms Reactor 워커 수 10 2 예상과 달리, CPU 코어 수(Parallel Scheduler Pool Size)에 따른 처리 시간 차이는 약 14ms(1.4%) 로 거의 없었습니다. 스레드 수가 2개에서 10개로 증가해도 성능 향상이 미미합니다. timeout/retry의 스케줄링 자체는 매우 가벼운 작업이므로, 스레드 수가 적어도 큰 영향을 주지 않는것을 확인할 수 있었습니다. 벤치 마크 코드 public class RunBenchmark { public static void main(String[] args) throws RunnerException { Options options = new OptionsBuilder() .include(OmsWebClientConcurrencyBenchmark.class.getSimpleName()) .forks(1) .warmupIterations(10) // 각 벤치 마크 실행 전 최적화를 위한 웜업 .warmupTime(TimeValue.seconds(5)) .measurementIterations(10) // 각 벤치마크 측정. 10회 반복 .measurementTime(TimeValue.seconds(5)) .timeUnit(TimeUnit.MILLISECONDS) .addProfiler(JavaFlightRecorderProfiler.class) // JFR 프로파일러 추가 .resultFormat(ResultFormatType.JSON) .result("jmh-results.json") .build(); new Runner(options).run(); } } @BenchmarkMode(Mode.AverageTime) public class OmsWebClientConcurrencyBenchmark { private static final int CONCURRENCY_CALL = 10; @State(Scope.Benchmark) public static class BenchState { @Param({"50"}) public int totalRequests; // 한 번의 벤치마크 호출당 요청 총 개수 @Param({"200"}) public int serverLatencyMs; // 목 서버가 응답 전 인위적으로 대기할 지연(ms) private DisposableServer server; private OmsService omsService; @Setup(Level.Trial) public void setup() { int availableProcessors = Runtime.getRuntime().availableProcessors(); String reactorPoolSize = System.getProperty("reactor.schedulers.defaultPoolSize", String.valueOf(availableProcessors)); System.out.println("\n========================================"); System.out.println("Benchmark started: OMS WebClient concurrency(10) under different CPU core caps"); System.out.println("Params: totalRequests=" + totalRequests + ", serverLatencyMs=" + serverLatencyMs); System.out.println("Available processors (current JVM): " + availableProcessors); System.out.println("Reactor Schedulers DefaultPoolSize: " + reactorPoolSize); System.out.println("========================================"); // 목 응답 JSON String json = "{" + "\"jibeonAddress\":{\"primaryAddress\":\"서울시 강남구\",\"secondaryAddress\":\"테스트로 123\",\"zipCode\":\"06200\"}," + "\"roadAddress\":{\"primaryAddress\":\"서울시 강남구\",\"secondaryAddress\":\"테스트로 123\",\"zipCode\":\"06200\"}," + "\"sigungu\":\"강남구\",\"dong\":\"역삼동\",\"provider\":\"OMS\",\"hcode\":\"11110\",\"bcode\":\"1111010100\",\"buildingNumber\":\"123\"" + "}"; this.server = HttpServer.create() .host("127.0.0.1") .port(0) .route(routes -> routes.post("/refine-address", (req, res) -> res.header("Content-Type", "application/json") .sendString(Mono.delay(Duration.ofMillis(serverLatencyMs)) .then(Mono.just(json))))) .bindNow(); int port = server.port(); WebClient omsClient = WebClient.builder().baseUrl("http://127.0.0.1:" + port).build(); WebClient fbkClient = WebClient.builder().baseUrl("http://127.0.0.1:" + port).build(); this.omsService = new OmsServiceImpl(omsClient, fbkClient); } @TearDown(Level.Trial) public void tearDown() { if (server != null) { server.disposeNow(); } } } private List<ApiResponse<RefineAddressDto>> invokeRefineAddressBatch(OmsService omsService, int totalRequests) { return Flux.range(0, totalRequests) .flatMap(i -> Mono.defer(() -> omsService.refineAddressIgnore("서울시 강남구", "테스트로 123") .defaultIfEmpty(ApiResponse.failure("NO_RESPONSE", false))), CONCURRENCY_CALL) .collectList() .block(); } // 코어 수 2로 실행 @Benchmark @Fork(value = 1, jvmArgsAppend = { "-XX:ActiveProcessorCount=2", "-XX:FlightRecorderOptions=threadbuffersize=16k,globalbuffersize=10m,memorysize=50m", "-XX:StartFlightRecording=settings=profile" }) public List<ApiResponse<RefineAddressDto>> refineAddress_concurrency10_cpu2(BenchState state, Blackhole blackhole) { var result = invokeRefineAddressBatch(state.omsService, state.totalRequests); blackhole.consume(result); return result; } // 코어 수 10으로 실행 @Benchmark @Fork(value = 1, jvmArgsAppend = { "-XX:ActiveProcessorCount=10", "-XX:FlightRecorderOptions=threadbuffersize=16k,globalbuffersize=10m,memorysize=50m", "-XX:StartFlightRecording=settings=profile" }) public List<ApiResponse<RefineAddressDto>> refineAddress_concurrency10_cpu10(BenchState state, Blackhole blackhole) { var result = invokeRefineAddressBatch(state.omsService, state.totalRequests); blackhole.consume(result); return result; } } 지연의 원인: Cold Start 원인을 분석하는 동안 유사한 지표를 가진 Trace를 추가로 수집할 수 있었는데요. 지표에서 한 가지 공통된 패턴을 발견할 수 있었습니다. 지연이 발생한 케이스 전부 어플리케이션 배포 직후 처음으로 주문이 등록되어 외부 API 호출이 발생했을때 인것을 확인할 수 있었습니다. Netty의 리소스는 Lazy Initialization 방식으로 동작합니다. 즉, WebClient를 생성하는 시점이 아니라 실제로 첫 번째 HTTP 요청을 보내는 시점에 초기화가 이루어집니다. Cold Start 해결책 두 가지 해결책을 검토했습니다. Connection Pool에 사전 커넥션 맺기: 미리 연결을 생성해 대기 Warmup 옵션: 리소스를 사전 로드하되 실제 연결은 필요 시점에 생성 Connection Pool에 사전 커넥션 생성 처음에는 애플리케이션 시작 시점에 미리 커넥션을 생성하여 풀에 대기시키는 방법을 고려하였습니다. 커넥션은 maxIdleTime, maxLifeTime 만큼 살아있다가 종료되는데, 기본값은 -1로 무제한입니다. 즉, 별도로 설정하지 않으면 커넥션이 시간 제한 없이 풀에 유지됩니다. 단, 두 설정값을 기본값(-1)으로 두면 서버 측의 keepAliveTimeout 설정과 충돌할 수 있습니다. 서버에서 커넥션을 먼저 끊으면 클라이언트는 이미 닫힌 커넥션으로 요청을 보내게 되어 "Connection reset by peer" 오류가 발생할 수 있습니다. 따라서 실무에서는 서버의 keepAliveTimeout보다 작은 값으로 maxIdleTime을 설정하는 것이 권장됩니다. 결국 애플리케이션 시작 시점에 미리 커넥션을 생성해 놓더라도 일정 시간 요청이 없으면 유휴 커넥션이 자동 해제되므로 트래픽이 간헐적인 '주문 등록'과 같은 케이스에는 적합하지 않습니다. Netty 이슈에서도 ConnectionPool을 웜업하는건 해결책이 아니며 고려사항이 아니라는것을 확인할 수 있습니다. (Add warmup functionality for the servers/clients #1455) Warmup 옵션 반면 warmup은 실제 TCP 커넥션을 맺지 않고 재사용 가능한 리소스만 사전 로드합니다. 따라서 커넥션이 해제되더라도 EventLoop, Selector, DNS Resolver 등은 이미 초기화되어 있어 후속 요청에서 초기화 비용이 발생하지 않습니다. @Configuration public class WebClientConfig { @Bean public WebClient omsWebClient() { HttpClient httpClient = HttpClient.create() .baseUrl("https://oms-api.example.com"); // 애플리케이션 시작 시 warmup 수행 httpClient.warmup().block(); return WebClient.builder() .clientConnector(new ReactorClientHttpConnector(httpClient)) .build(); } } 마무리 이 과정을 통해 단순히 라이브러리를 사용하는 것을 넘어, 내부 동작 원리를 이해하고 적절한 설정을 선택하는 것의 중요성을 배웠습니다.

dev.to

IA DLC Con Kiro - Parte 1

Después de mi participación como speaker en varios AWS Community Days durante el 2025, hablando de cómo potenciar procesos DevOps con Amazon Q Developer y CodeCatalyst, y asistir al AWS re:Invent 2025, confirmé lo que venía transmitiendo en mis charlas: Los desarrolladores no vamos a desaparecer. Es una era de cambio en donde vamos a tener a nuestra disposición un nuevo conjunto de herramientas que nos ayudarán a ser más productivos y entregar mejores soluciones. Como dijo Vogels en su Keynote: Will AI make obsolete, NO (if you evolve) Una de esas herramientas es Amazon Kiro, que emerge como un IDE que promete hacer "vibe coding" para prototipar rápidamente, pero también llevar la ingeniería de software a otro nivel con un enfoque de Spec Driven Development cuando quieres llevar tu prototipo a producción y que sea escalable, seguro y confiable. Si aun no tienes claro que es Kiro te invito a ver mi charla virtual con el AWS User Group de Cúcuta: https://www.youtube.com/watch?v=_6a4oSl8LeY En mi charla en el AWS Community Day Ecuador 2025 estuve hablando de cómo implementar agentes para cada etapa del SDLC con Amazon Q Developer CLI. Ahora, con la actualización a Kiro CLI, veamos cómo podemos implementar agentes para un flujo completo de desarrollo de software. Las Etapas del SDLC con Agentes Veamos entonces las etapas básicas del SDLC que podemos abordar con agentes. Si tienes más etapas en tu proceso de desarrollo, puedes implementar un agente específico para cada una. Etapa 1: Análisis de Requisitos Objetivo: Extraer y estructurar requisitos del negocio Agente: requirements-analyst Entregable: Especificaciones técnicas estructuradas Etapa 2: Diseño del Sistema Objetivo: Crear arquitectura y diseño técnico Agente: system-architect Entregable: Diagramas de arquitectura, esquemas de BD, APIs Etapa 3: Implementación Objetivo: Generar código production-ready Agente: backend-developer Entregable: Código fuente, componentes funcionales Etapa 4: Pruebas Objetivo: Validar calidad y funcionalidad Agente: qa-engineer Entregable: Suite de tests completa Etapa 5: Despliegue Objetivo: Automatizar deployment a producción Agente: devops-engineer Entregable: Pipelines CI/CD, infraestructura como código Etapa 6: Mantenimiento Objetivo: Monitorear y optimizar en producción Agente: sre-specialist Entregable: Dashboards, alertas, runbooks Dividiremos esta entrega en varios capitulos, en este primer capitulo abordaremos los agentes que nos ayudaran a documentar nuestro proyecto: requirements-analyst y system-architect. En la próxima entrega abordaremos los ejecutores: backend-developer y qa-engineer, finalizaremos con devops-engineer y sre-specialist.No te pierdas esta serie que va a estar interesante. Estructura de un Agente en Kiro CLI Teniendo claro las etapas, ahora miremos la estructura de un agente en Kiro CLI. Anatomía de un Agente Todo agente se configura mediante un archivo JSON con esta estructura: { "name": "nombre-del-agente", "description": "Propósito y capacidades del agente", "model": "claude-sonnet-4", "mcp_servers": { "servidor": { "command": "npx", "args": ["-y", "@paquete/servidor"], "env": { "TOKEN": "${VARIABLE_ENTORNO}" } } }, "tools": [ { "type": "web_search_20250305", "name": "web_search" } ] } Componentes Clave name: Identificador único (usa kebab-case) "name": "nombre-del-agente", description: Qué hace y cuándo usarlo "description": "Propósito y capacidades del agente", model: Modelo de Claude a usar "model": "claude-sonnet-4" mcp_servers: Servidores MCP para extender capacidades "mcp_servers": { "servidor": { "command": "npx", "args": ["-y", "@paquete/servidor"], "env": { "TOKEN": "${VARIABLE_ENTORNO}" } } }, tools: Herramientas adicionales "tools": [ { "type": "web_search_20250305", "name": "web_search" } ] *Manos a la Obra: Definamos Nuestros Agentes * Preparación del Entorno 1. Crear directorio de configuraciones (Si no existe, verifica que el directorio no ha sido creado) mkdir -p ~/.kiro/agents 2. Configurar variables de entorno export AWS_ACCESS_KEY_ID="tu_access_key" export AWS_SECRET_ACCESS_KEY="tu_secret_key" Nota: Aca puedes configurar cualquier variable de entorno que sea necesaria para los agentes. Agente 1: Requirements Analyst Propósito: Analizar requisitos de negocio y generar especificaciones técnicas. Configuración Crear archivo: ~/.kiro/agents/requirements-analyst.json { "name": "requirements-analyst", "description": "Especialista en análisis de requisitos", "prompt": "Especialista en análisis de requisitos. Extrae requisitos funcionales y no funcionales de conversaciones, documentos y reuniones. Genera especificaciones técnicas claras en formato de Historias de usuario, que cumplen con el formato INVEST. Debes crear un archivo en formato MD de las HU's sobre el directorio donde estas ubicado.", "mcpServers": {}, "tools": [ "*" ], "toolAliases": {}, "allowedTools": [], "resources": [ "file://AGENTS.md", "file://README.md" ], "hooks": {}, "toolsSettings": {}, "useLegacyMcpJson": true, "model": "claude-haiku-4.5" } Uso kiro-cli chat --agent requirements-analyst "Define los requisitios para construir una API Restful con FastAPI para el manejo de tareas" Salida Esperada # Historias de Usuario - API RESTful FastAPI para Manejo de Tareas ## Requisitos Funcionales ### HU-001: Crear una nueva tarea **Como** usuario de la API **Quiero** crear una nueva tarea con título y descripción **Para** registrar mis tareas pendientes **Criterios de Aceptación:** - El endpoint POST `/tasks` debe aceptar título y descripción - La tarea debe asignarse un ID único automáticamente - La tarea debe tener estado inicial "pendiente" - Retornar código 201 con la tarea creada - Validar que título no esté vacío (mínimo 3 caracteres) **Tamaño:** M | **Prioridad:** Alta | **Sprint:** 1 --- ### HU-002: Listar todas las tareas **Como** usuario de la API **Quiero** obtener el listado completo de tareas **Para** visualizar todas mis tareas registradas **Criterios de Aceptación:** - El endpoint GET `/tasks` retorna lista de todas las tareas - Incluir paginación (limit, offset) - Retornar código 200 con array de tareas - Soportar filtrado por estado (pendiente, completada, cancelada) - Ordenar por fecha de creación descendente por defecto **Tamaño:** M | **Prioridad:** Alta | **Sprint:** 1 Agente 2: System Architect Propósito: Diseñar arquitectura del sistema y componentes técnicos. Configuración Crear archivo: ~/.kiro/agents/system-architect.json { "name": "system-architect", "description": "Arquitecto de AWS para diseñar soluciones de arquitectura de AWS", "prompt": "Eres un arquitecto de soluciones de AWS especializado en diseñar arquitecturas escalables, seguras y eficientes en costos. Tu objetivo es ayudar a los usuarios a crear soluciones de arquitectura de AWS que sigan las mejores prácticas del Well-Architected Framework. Proporciona recomendaciones específicas sobre servicios de AWS, patrones de arquitectura, consideraciones de seguridad, optimización de costos y estrategias de escalabilidad. Siempre considera los cinco pilares del Well-Architected Framework: excelencia operacional, seguridad, confiabilidad, eficiencia de rendimiento y optimización de costos.", "mcpServers": { "awslabs.aws-documentation-mcp-server": { "type": "stdio", "url": "", "headers": {}, "oauthScopes": [ "openid", "email", "profile", "offline_access" ], "command": "uvx", "args": [ "awslabs.aws-documentation-mcp-server@latest" ], "env": { "FASTMCP_LOG_LEVEL": "ERROR" }, "timeout": 120000, "disabled": false }, "awslabs.aws-diagram-mcp-server": { "type": "stdio", "url": "", "headers": {}, "oauthScopes": [ "openid", "email", "profile", "offline_access" ], "command": "uvx", "args": [ "awslabs.aws-diagram-mcp-server@latest" ], "env": { "FASTMCP_LOG_LEVEL": "ERROR" }, "timeout": 120000, "disabled": false } }, "tools": [ "*" ], "toolAliases": {}, "allowedTools": [], "resources": [ "file://AGENTS.md", "file://README.md" ], "hooks": {}, "toolsSettings": {}, "useLegacyMcpJson": true, "model":"claude-haiku-4.5" } Para este agente vamos a utilizar los MCP oficiales de AWS para documentación y diagramación, para ello debes asegurarte de tener instalado en tu entorno: python uv grahpviz Puedes guiarte de este articulo: https://aws.amazon.com/es/blogs/machine-learning/build-aws-architecture-diagrams-using-amazon-q-cli-and-mcp/ Uso Inicializa tu agente kiro-cli chat --agent system-architect Asegurate que los MCP's configurados para el agente se inicialicen correctamente: Envia el prompt para generar tu arquitectura, documentarla y diagramarla Diseña y documenta (genera un archivo md), la arquitectura para un API Restful escrita en python con FastAPI que debe ser desplegada en un modelo serverless en AWS, ten en cuenta las mejores practicas y que sea costo eficiente, el diseño de arquitectura debe incluir todos los componenentes necesarios. Para lograr el objetivo debes consultar la documentación oficial de AWS y tambien debes crear un diagrama de la arquitectura propuesta Salida Esperada Diagrama de Arquitectura Documentación de la arquitectura Como puedes ver la configuración de agentes especializados te va a ayudar en tu SDLC, dejando todo documentado y siguiendo las buenas practicas. Hasta aquí el primer capitulo de esta serie, debes estar atento al proximo capitulo donde abordaremos los agentes de desarrollo y pruebas. Nos vemos en el próximo capitulo. Puedes consultar la documentación oficial de kiro en: https://kiro.dev/docs/ Novedades de AWS re:Invent 2025 Si quieres estar al tanto de todas las novedades de AWS puedes consultarlas aca: https://aws.amazon.com/es/blogs/aws/top-announcements-of-aws-reinvent-2025/ También puedes tomar este learning path en Skill Builder: https://skillbuilder.aws/learning-plan/JZQY2Z8DG4/aws-reinvent-2025-announcements-learning-plan/VWQU3VK65K

dev.to

node-cron for scheduled jobs

# Scheduling Tasks in Node.js: A Comprehensive Guide with Practical Examples In backend development, we frequently encounter the need to execute tasks at specific times or at regular intervals. Whether it's for sending reminder emails, generating daily reports, cleaning temporary data, or synchronizing information, task automation is crucial for the efficiency and robustness of any application. This post will guide you through the process of installing and using cron libraries in Node.js, demonstrating how to create recurring tasks with practical examples, focusing on a reminder scenario. Why Automate Tasks? Automating repetitive tasks frees up valuable resources, both human and computational. Instead of relying on manual intervention, which is prone to errors and time-consuming, we can depend on automated systems to perform actions reliably and punctually. In web applications, this translates to a better user experience, reduced server load, and increased scalability. Cron Libraries in Node.js: Introduction The term \"cron\" originates from Unix, where crontab is the standard utility for scheduling commands. In the Node.js ecosystem, several libraries replicate this functionality, offering a user-friendly interface for defining and managing scheduled tasks. Among the most popular are: node-cron: A lightweight and flexible library with a syntax inspired by Unix cron. agenda: A more robust solution that uses MongoDB for job persistence, ideal for production applications requiring reliability. In this guide, we will focus on the node-cron library due to its simplicity and ease of use for most cases. Installing node-cron To get started, ensure you have Node.js and npm (or yarn) installed on your environment. Then, install the node-cron library in your project: npm install node-cron # or yarn add node-cron If you are using TypeScript, also install the types for the library: npm install --save-dev @types/node-cron # or yarn add --dev @types/node-cron Creating Recurring Tasks with node-cron The basic syntax of node-cron allows you to schedule tasks using a cron string, which specifies the execution pattern. The string consists of five fields (or six, depending on configuration, including seconds): * * * * * * ┬ ┬ ┬ ┬ ┬ ┬ │ │ │ │ │ │ │ │ │ │ │ └─ Day of the Week (0 - 7) (Sunday is 0 or 7) │ │ │ │ └─ Month (1 - 12) │ │ │ └─ Day of the Month (1 - 31) │ │ └─ Hour (0 - 23) │ └─ Minute (0 - 59) └─ Second (0 - 59) (optional, defaults to 0 if not included) Each field can contain: *: Matches any value (e.g., * in minutes means \"every minute\"). */n: Executes every n units (e.g., */15 in minutes means \"every 15 minutes\"). n: A specific value (e.g., 0 in hours means \"at midnight\"). n-m: A range of values (e.g., 9-17 in hours means \"from 9 AM to 5 PM\"). n,m,p: A list of values (e.g., 1,3,5 in days of the week means \"Monday, Wednesday, and Friday\"). Example: Task Reminders Let's create a practical example where we want to schedule reminders for users about pending tasks. Imagine we have a ReminderService that sends notifications. First, let's define an interface for the task and the service: // src/interfaces/task.interface.ts export interface Task { id: string; userId: string; description: string; dueDate: Date; reminderSent?: boolean; } // src/services/reminder.service.ts import { Task } from '../interfaces/task.interface'; class ReminderService { private tasks: Task[] = []; // Simulates an in-memory database constructor() { // Populating with some example tasks this.tasks.push({ id: 'task-1', userId: 'user-123', description: 'Finalize monthly report', dueDate: new Date(Date.now() + 2 * 60 * 60 * 1000), // In 2 hours }); this.tasks.push({ id: 'task-2', userId: 'user-456', description: 'Schedule team meeting', dueDate: new Date(Date.now() + 24 * 60 * 60 * 1000), // In 24 hours }); this.tasks.push({ id: 'task-3', userId: 'user-123', description: 'Review code for feature X', dueDate: new Date(Date.now() + 2 * 24 * 60 * 60 * 1000), // In 2 days reminderSent: true, // Reminder already sent }); } /** * Fetches tasks that need a reminder. * @returns Array of tasks needing a reminder. */ getTasksNeedingReminder(): Task[] { console.log('Fetching tasks for reminder...'); return this.tasks.filter(task => !task.reminderSent && // Reminder not yet sent task.dueDate > new Date() && // Due date is in the future (task.dueDate.getTime() - Date.now()) <= (24 * 60 * 60 * 1000) // Due date is within the next 24 hours ); } /** * Simulates sending a reminder for a specific task. * @param task The task for which the reminder will be sent. */ sendReminder(task: Task): void { console.log(`Sending reminder to user ${task.userId} about task \"${task.description}\".`); // Here you would implement the actual logic for sending emails, push notifications, etc. // Marking the task as reminder sent to avoid duplicate sends const taskIndex = this.tasks.findIndex(t => t.id === task.id); if (taskIndex !== -1) { this.tasks[taskIndex].reminderSent = true; } } } export default new ReminderService(); Now, let's integrate node-cron to trigger the reminder check and sending. We'll create a scheduler.ts script responsible for managing the scheduled tasks. // src/scheduler.ts import cron from 'node-cron'; import reminderService from './services/reminder.service'; console.log('Initializing task scheduler...'); // Schedule the task to run every hour, at the 0th minute. // The pattern '0 * * * *' means: at minute 0 of every hour, every day, every month, every day of the week. const reminderCronJob = cron.schedule('0 * * * *', () => { console.log('Running reminder check job...'); const tasksToRemind = reminderService.getTasksNeedingReminder(); if (tasksToRemind.length > 0) { console.log(`Found ${tasksToRemind.length} tasks to send reminders for.`); tasksToRemind.forEach(task => { reminderService.sendReminder(task); }); } else { console.log('No tasks found for reminders in this run.'); } }, { scheduled: true, timezone: \"America/Sao_Paulo\" // Set your desired timezone }); // Optional: Add a listener for job events reminderCronJob.start(); // Ensure the job starts reminderCronJob.stop(); // Stop the job (example) reminderCronJob.setTime(new cron.CronTime('*/5 * * * * *')); // Change the schedule to every 5 seconds (for demonstration only) reminderCronJob.start(); // Restart with the new schedule console.log('Task scheduler started. Checking for reminders every hour.'); // To keep the process running (in a real scenario, your Node.js server would already be running) process.stdin.resume(); process.on('SIGINT', () => { console.log('Received SIGINT. Stopping scheduler and exiting...'); reminderCronJob.stop(); process.exit(0); }); Code Explanation: import cron from 'node-cron';: Imports the node-cron library. import reminderService from './services/reminder.service';: Imports our service class containing the business logic. cron.schedule('0 * * * *', () => { ... }, { ... });: This is the main function. The first argument ('0 * * * *') is the cron string that defines when the callback function will be executed (in this case, every hour on the hour). The second argument is the callback function that will be executed at the scheduled time. Inside it, we call getTasksNeedingReminder from our service and iterate over the found tasks to send reminders. The third argument is an options object: scheduled: true: Indicates that the job should start automatically when defined. timezone: \"America/Sao_Paulo\": Essential to ensure that times are interpreted correctly according to your application's timezone. reminderCronJob.start();: Starts the execution of the scheduled job. reminderCronJob.stop();: Allows you to stop the job's execution. reminderCronJob.setTime(...): Demonstrates how you can change the schedule of an existing job. process.stdin.resume();: In a simple script like this, we use this to keep the Node.js process running, as cron.schedule runs in the background. In a real web server (like Express), the server itself keeps the process active. process.on('SIGINT', ...): Basic handling for the interrupt signal (Ctrl+C), ensuring the job is stopped gracefully before the process exits. To run this script, save it as scheduler.ts, compile it to JavaScript (if using TypeScript), and run it with Node.js: tsc # If using TypeScript node dist/scheduler.js # Or the path to your compiled JS file You will see logs indicating when the reminder check job runs and when reminders are sent. Best Practices and Considerations Timezone: Always set the timezone correctly in cron.schedule to avoid surprises with execution times. State Management: In robust applications, the state of jobs (like \"reminder sent") should be persisted in a database, not in memory, to ensure tasks are not lost if the server restarts. Error Handling: Implement robust try...catch blocks within the job callback functions to catch and log any errors that may occur during execution, preventing a failed job from interrupting others. Concurrency: If a task takes longer to execute than the interval between runs, you might encounter concurrency issues. Libraries like agenda offer mechanisms to handle this. For node-cron, you can implement your own locking logic. Schedule Complexity: Avoid overly complex cron strings. If the scheduling logic is too intricate, consider alternative approaches or simplify the schedule and add additional logic within the callback. Monitoring: Monitor the execution of your scheduled jobs. Logging and monitoring tools are essential to ensure everything is working as expected. Conclusion Scheduling recurring tasks in Node.js is a powerful technique for automating processes and improving the efficiency of your applications. With libraries like node-cron, the process becomes accessible and flexible. By understanding cron syntax, implementing business logic in an organized manner, and following best practices, you can create robust systems that reliably execute important actions, from sending reminders to maintaining complex routines. Remember to adapt the examples and considerations to the specific complexity and requirements of your project.

dev.to

node-cron para jobs agendados

## Agendando Tarefas em Node.js: Um Guia Completo com Exemplos Práticos No desenvolvimento backend, frequentemente nos deparamos com a necessidade de executar tarefas em horários específicos ou em intervalos regulares. Seja para enviar e-mails de lembrete, gerar relatórios diários, limpar dados temporários ou sincronizar informações, a automação de tarefas é crucial para a eficiência e robustez de qualquer aplicação. Este post irá guiá-lo através do processo de instalação e utilização de bibliotecas de cron em Node.js, demonstrando como criar tarefas recorrentes com exemplos práticos, focando em um cenário de lembretes. Por Que Automatizar Tarefas? A automação de tarefas repetitivas libera recursos valiosos, tanto humanos quanto computacionais. Em vez de depender de intervenção manual, que é propensa a erros e demorada, podemos confiar em sistemas automatizados para executar ações de forma confiável e pontual. Em aplicações web, isso se traduz em melhor experiência do usuário, menor sobrecarga de servidores e maior escalabilidade. Bibliotecas de Cron em Node.js: Introdução O termo \"cron\" origina-se do Unix, onde o crontab é o utilitário padrão para agendar comandos. No ecossistema Node.js, diversas bibliotecas replicam essa funcionalidade, oferecendo uma interface amigável para definir e gerenciar tarefas agendadas. Dentre as mais populares, destacam-se: node-cron: Uma biblioteca leve e flexível, com sintaxe inspirada no cron do Unix. agenda: Uma solução mais robusta, que utiliza MongoDB para persistência de jobs, ideal para aplicações em produção que exigem confiabilidade. Neste guia, focaremos na biblioteca node-cron devido à sua simplicidade e facilidade de uso para a maioria dos casos. Instalação do node-cron Para começar, certifique-se de ter o Node.js e o npm (ou yarn) instalados em seu ambiente. Em seguida, instale a biblioteca node-cron em seu projeto: npm install node-cron # ou yarn add node-cron Se você estiver utilizando TypeScript, instale também os tipos para a biblioteca: npm install --save-dev @types/node-cron # ou yarn add --dev @types/node-cron Criando Tarefas Recorrentes com node-cron A sintaxe básica do node-cron permite agendar tarefas usando uma string de cron, que especifica o padrão de execução. A string é composta por cinco campos (ou seis, dependendo da configuração, incluindo segundos): * * * * * * ┬ ┬ ┬ ┬ ┬ ┬ │ │ │ │ │ │ │ │ │ │ │ └─ Dia da Semana (0 - 7) (Domingo é 0 ou 7) │ │ │ │ └─ Mês (1 - 12) │ │ │ └─ Dia do Mês (1 - 31) │ │ └─ Hora (0 - 23) │ └─ Minuto (0 - 59) └─ Segundo (0 - 59) (opcional, se não incluído, padrão é 0) Cada campo pode conter: *: Corresponde a qualquer valor (ex: * em minutos significa \"a cada minuto\"). */n: Executa a cada n unidades (ex: */15 em minutos significa \"a cada 15 minutos\"). n: Um valor específico (ex: 0 em hora significa \"à meia-noite\"). n-m: Um intervalo de valores (ex: 9-17 em horas significa \"das 9h às 17h\"). n,m,p: Uma lista de valores (ex: 1,3,5 em dias da semana significa \"segunda, quarta e sexta\"). Exemplo: Lembretes de Tarefas Vamos criar um exemplo prático onde queremos agendar lembretes para usuários sobre tarefas pendentes. Imagine que temos um serviço ReminderService que envia notificações. Primeiro, vamos definir uma interface para a tarefa e o serviço: // src/interfaces/task.interface.ts export interface Task { id: string; userId: string; description: string; dueDate: Date; reminderSent?: boolean; } // src/services/reminder.service.ts import { Task } from '../interfaces/task.interface'; class ReminderService { private tasks: Task[] = []; // Simula um banco de dados em memória constructor() { // Populando com algumas tarefas de exemplo this.tasks.push({ id: 'task-1', userId: 'user-123', description: 'Finalizar relatório mensal', dueDate: new Date(Date.now() + 2 * 60 * 60 * 1000), // Daqui a 2 horas }); this.tasks.push({ id: 'task-2', userId: 'user-456', description: 'Agendar reunião com a equipe', dueDate: new Date(Date.now() + 24 * 60 * 60 * 1000), // Daqui a 24 horas }); this.tasks.push({ id: 'task-3', userId: 'user-123', description: 'Revisar código da feature X', dueDate: new Date(Date.now() + 2 * 24 * 60 * 60 * 1000), // Daqui a 2 dias reminderSent: true, // Lembrete já enviado }); } /** * Busca tarefas que precisam de lembrete. * @returns Array de tarefas que precisam de lembrete. */ getTasksNeedingReminder(): Task[] { console.log('Buscando tarefas para lembrete...'); return this.tasks.filter(task => !task.reminderSent && // Lembrete ainda não foi enviado task.dueDate > new Date() && // Data de vencimento é no futuro (task.dueDate.getTime() - Date.now()) <= (24 * 60 * 60 * 1000) // Data de vencimento é nas próximas 24h ); } /** * Simula o envio de um lembrete para uma tarefa específica. * @param task A tarefa para a qual o lembrete será enviado. */ sendReminder(task: Task): void { console.log(`Enviando lembrete para o usuário ${task.userId} sobre a tarefa \"${task.description}\".`); // Aqui você implementaria a lógica real de envio de e-mail, push notification, etc. // Marcando a tarefa como lembrete enviado para evitar envios duplicados const taskIndex = this.tasks.findIndex(t => t.id === task.id); if (taskIndex !== -1) { this.tasks[taskIndex].reminderSent = true; } } } export default new ReminderService(); Agora, vamos integrar o node-cron para disparar a verificação e o envio de lembretes. Criaremos um script scheduler.ts que será responsável por gerenciar as tarefas agendadas. // src/scheduler.ts import cron from 'node-cron'; import reminderService from './services/reminder.service'; console.log('Inicializando o agendador de tarefas...'); // Agendando a tarefa para rodar a cada hora, no minuto 0. // O padrão '0 * * * *' significa: no minuto 0 de cada hora, de cada dia, de cada mês, de cada dia da semana. const reminderCronJob = cron.schedule('0 * * * *', () => { console.log('Executando job de verificação de lembretes...'); const tasksToRemind = reminderService.getTasksNeedingReminder(); if (tasksToRemind.length > 0) { console.log(`Encontradas ${tasksToRemind.length} tarefas para enviar lembretes.`); tasksToRemind.forEach(task => { reminderService.sendReminder(task); }); } else { console.log('Nenhuma tarefa encontrada para lembretes nesta execução.'); } }, { scheduled: true, timezone: \"America/Sao_Paulo\" // Defina o fuso horário desejado }); // Opcional: Adicionar um listener para eventos do job reminderCronJob.start(); // Garante que o job inicie reminderCronJob.stop(); // Para o job (exemplo) reminderCronJob.setTime(new cron.CronTime('*/5 * * * * *')); // Altera o schedule para a cada 5 segundos (apenas para demonstração) reminderCronJob.start(); // Reinicia com o novo schedule console.log('Agendador de tarefas iniciado. Verificando lembretes a cada hora.'); // Para manter o processo rodando (em um cenário real, seu servidor Node.js já estaria rodando) process.stdin.resume(); process.on('SIGINT', () => { console.log('Recebido SIGINT. Parando o agendador e encerrando...'); reminderCronJob.stop(); process.exit(0); }); Explicação do Código: import cron from 'node-cron';: Importa a biblioteca node-cron. import reminderService from './services/reminder.service';: Importa nossa classe de serviço que contém a lógica de negócio. cron.schedule('0 * * * *', () => { ... }, { ... });: Esta é a função principal. O primeiro argumento ('0 * * * *') é a string de cron que define quando a função de callback será executada (neste caso, a cada hora cheia). O segundo argumento é a função de callback que será executada no horário agendado. Dentro dela, chamamos getTasksNeedingReminder do nosso serviço e iteramos sobre as tarefas encontradas para enviar os lembretes. O terceiro argumento é um objeto de opções: scheduled: true: Indica que o job deve iniciar automaticamente quando definido. timezone: \"America/Sao_Paulo\": Essencial para garantir que os horários sejam interpretados corretamente de acordo com o fuso horário da sua aplicação. reminderCronJob.start();: Inicia a execução do job agendado. reminderCronJob.stop();: Permite parar a execução do job. reminderCronJob.setTime(...): Demonstra como é possível alterar o schedule de um job existente. process.stdin.resume();: Em um script simples como este, usamos isso para manter o processo Node.js em execução, pois o cron.schedule roda em segundo plano. Em um servidor web real (como Express), o servidor já mantém o processo ativo. process.on('SIGINT', ...): Um tratamento básico para o sinal de interrupção (Ctrl+C), garantindo que o job seja parado graciosamente antes de o processo encerrar. Para executar este script, salve-o como scheduler.ts, compile-o para JavaScript (se estiver usando TypeScript) e execute-o com Node.js: tsc # Se estiver usando TypeScript node dist/scheduler.js # Ou o caminho para o seu arquivo JS compilado Você verá os logs indicando quando o job de verificação de lembretes é executado e quando os lembretes são enviados. Boas Práticas e Considerações Fuso Horário: Sempre defina o fuso horário (timezone) corretamente em cron.schedule para evitar surpresas com horários de execução. Gerenciamento de Estado: Em aplicações robustas, o estado dos jobs (como \"lembrete enviado") deve ser persistido em um banco de dados, não em memória, para garantir que as tarefas não sejam perdidas em caso de reinicialização do servidor. Tratamento de Erros: Implemente blocos try...catch robustos dentro das funções de callback dos jobs para capturar e logar quaisquer erros que possam ocorrer durante a execução, evitando que um job falho interrompa outros. Concorrência: Se uma tarefa demorar mais para executar do que o intervalo entre as execuções, você pode ter problemas de concorrência. Bibliotecas como agenda oferecem mecanismos para lidar com isso. Para node-cron, você pode implementar sua própria lógica de bloqueio. Complexidade do Schedule: Evite strings de cron excessivamente complexas. Se a lógica de agendamento for muito intrincada, considere abordagens alternativas ou simplifique o schedule e adicione lógica adicional dentro do callback. Monitoramento: Monitore a execução dos seus jobs agendados. Ferramentas de logging e monitoramento são essenciais para garantir que tudo esteja funcionando como esperado. Conclusão Agendar tarefas recorrentes em Node.js é uma técnica poderosa para automatizar processos e melhorar a eficiência de suas aplicações. Com bibliotecas como node-cron, o processo se torna acessível e flexível. Ao entender a sintaxe de cron, implementar a lógica de negócio de forma organizada e seguir boas práticas, você pode criar sistemas robustos que executam ações importantes de forma confiável, desde o envio de lembretes até a manutenção de rotinas complexas. Lembre-se de adaptar os exemplos e considerações à complexidade e aos requisitos específicos do seu projeto.

dev.to

Cyberpunk 2077: Ultimate Edition and other Switch games are up to 50 percent off

Unlike the day after Thanksgiving, when people are rushing online and to the stores to look for deals, the day after Christmas is usually a time to recover from the festivities and check your budget to see how much all that gifting has cost you. However, if you're in a shopping mood, or if you [&#8230;]

theverge.com

Times Square to feature patriotic ball drop for New Year's Eve

The New Year's Eve ball drop in New York City will sparkle in red, white, and blue to kick off celebrations for the U.S.'s 250th birthday in 2026

abcnews.go.com

Google is letting some people change their @gmail address

If you've been saddled with some awful Gmail address you chose in high school (looking at you, LazySexyCool3030), help appears to be on the way. According to a Google account support page in Hindi the company is allowing some people to change their email address. According to the page (via Google Translate): If your Google [&#8230;]

theverge.com

The EKS 1.32 1.33 Upgrade That Broke Everything (And How I Fixed It)

Introduction Upgrading Kubernetes should be boring. This one wasn’t. I recently upgraded a production Amazon EKS cluster from 1.32 to 1.33, expecting a routine change. Instead, it triggered a cascading failure: Nodes went NotReady Add-ons stalled indefinitely Karpenter stopped provisioning capacity The cluster deadlocked itself This post walks through what broke, why it broke, and the exact steps that stabilized the cluster so you don’t repeat my mistakes. Critical Issues If you're upgrading to EKS 1.33, know this: Amazon Linux 2 is NOT supported - Must migrate to AL2023 first Anonymous auth is restricted - New RBAC required for kube-apiserver Karpenter needs eks:DescribeCluster permission - Missing this breaks everything Addons can get stuck "Updating" - managed node groups are your escape hatch Part 1: The Failed First Attempt What I Did Wrong I started with what looked like a standard Terraform upgrade: module "damola_eks" { source = "terraform-aws-modules/eks/aws" version = "~> 20.0" cluster_name = local.name cluster_version = "1.33" What happened Error: AMI Type AL2_x86_64 is only supported for kubernetes versions 1.32 or earlier The Root Cause EKS 1.33 drops Amazon Linux 2 completely: AL2 reaches end of support on Nov 26, 2025 and no AL2 AMIs exist for 1.33. The Fix: AL2 → AL2023 Migration For Karpenter users, this is actually simple. Update your EC2NodeClass: # EC2NodeClass spec: amiSelectorTerms: - alias: al2023@latest Managed node groups (Terraform) ami_type = "AL2023_x86_64_STANDARD" Wait until all nodes are AL2023, then upgrade the control plane. Part 2: The Karpenter Catastrophe After migrating to AL2023, I cordoned old nodes, no new nodes came up. Karpenter was completely stuck. The Error Checking Karpenter logs revealed: { "message": "failed to detect the cluster CIDR", "error": "not authorized to perform: eks:DescribeCluster" } The Root Cause Starting with Karpenter v1.0, the controller requires eks:DescribeCluster to: Detect cluster networking (CIDR) Discover API endpoint configuration Validate authentication mode Without this permission, provisioning silently fails. The Fix Add the permission to your Karpenter controller IAM role: { "Effect": "Allow", "Action": "eks:DescribeCluster", "Resource": "*" } Then restart: kubectl rollout restart deployment/karpenter -n karpenter kubectl rollout status deployment/karpenter -n karpenter Karpenter recovered but the cluster still wasn’t healthy. Part 3: The Addon Deadlock After the control plane upgraded: Add-ons started updating (vpc-cni, kube-proxy, coredns) They got stuck in Updating All nodes went NotReady No new nodes could join Classic deadlock: Nodes need add-ons → add-ons need healthy nodes. The Error kubectl get nodes # All showed: NotReady kubectl logs -n kube-system -l k8s-app=kube-dns # Error: Authorization error (user=kube-apiserver-kubelet-client, verb=get, resource=nodes, subresource=proxy) The Root Causes Anonymous auth restricted (EKS 1.33) Anonymous API access is now limited to health endpoints only. The kube-apiserver requires explicit RBAC to communicate with kubelet. Add-on update deadlock Add-ons need healthy nodes to update. Nodes need working add-ons to become Ready. When all nodes are NotReady, everything gets stuck. The Fix Part 1: RBAC for kube-apiserver Create the missing RBAC: kubectl apply -f - <<EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:kube-apiserver-to-kubelet rules: - apiGroups: [""] resources: ["nodes/proxy","nodes/stats","nodes/log","nodes/spec","nodes/metrics"] verbs: ["*"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:kube-apiserver roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:kube-apiserver-to-kubelet subjects: - kind: User name: kube-apiserver-kubelet-client apiGroup: rbac.authorization.k8s.io EOF Errors stopped but add-ons were still stuck. The Fix Part 2: Breaking the Deadlock with Managed Nodes With broken Karpenter nodes, I had no way out. Solution: temporarily scale up managed node groups. desired_size = 1 ami_type = "AL2023_x86_64_STANDARD" Why this works Managed nodes bootstrap independently They come up with working VPC CNI Add-ons get healthy replicas Karpenter recovers Broken nodes can be safely deleted Within ~10 minutes, the cluster recovered. Part 4: Final Cleanup & Validation Once stable verify all nodes healthy: kubectl get nodes -o custom-columns=\ NAME:.metadata.name,\ STATUS:.status.conditions[-1].type,\ OS:.status.nodeInfo.osImage,\ VERSION:.status.nodeInfo.kubeletVersion # All showed: # Ready | Amazon Linux 2023.9.20251208 | v1.33.5-eks-ecaa3a6 # Verify addons kubectl get daemonset -n kube-system # All showed READY = DESIRED # Clean up stuck terminating pods kubectl delete pod -n kube-system --force --grace-period=0 <stuck-pod-names> Recommended Add-on Versions for EKS 1.33 CoreDNS: v1.12.4-eksbuild.1 kube-proxy: v1.33.5-eksbuild.2 VPC CNI: v1.21.1-eksbuild.1 EBS CSI: v1.54.0-eksbuild.1 Key Takeaways AL2023 is mandatory for EKS 1.33 Karpenter needs eks:DescribeCluster kube-apiserver RBAC must be updated Keep managed node groups as a safety net Note Looking back, the main issue wasn’t just missing permissions it was configuration drift. While the cluster was still running EKS 1.32, I manually added eks:DescribeCluster during the AL2023 migration. Everything worked, so I forgot to codify it in Terraform. During the upgrade to EKS 1.33, Terraform re-applied the IAM role and removed the permission right when Karpenter started requiring it. The upgrade didn’t introduce the bug. Environment: EKS, Terraform, Karpenter v1.x Resources AWS EKS Version Documentation Amazon Linux 2023 Migration Guide EKS Kubernetes Versions (Standard Support)

dev.to

Do Anti-Vibecoding ao Caos: A Saga do "Ficha Monstra"

O projeto "Ficha Monstra" nasceu de uma mistura de necessidade e impulso: eu comprei um VPS durante a black friday por puro consumismo, 2) eu queria escrever e aprender mais sobre usar a inteligência artificial de uma maneira legal no desenvolvimento de software e 3) eu estava cansado de perder minhas fichas de papel na caixinha de fichas de treinos na academia. Ai nasceu o Ficha Monstra, nome que veio do gemini. A premissa era criar uma stack "anti-vibecoding": queria estrutura e controle. Escolhi Blazor .NET Core, Identity, EF e Semantic Kernel no Azure. A Fase da Ordem Comecei disciplinado. Usei o Gemini apenas para brainstorming e geração de HTML/CSS, mas mantive o controle arquitetural. Criei a solução .NET manualmente (para evitar o vício das IAs em bibliotecas antigas) e usei o Spec Kit para criar uma "Constituição" do projeto, impondo regras rígidas ao modelo. O resultado? Um MVP honesto e funcional entregue em tempo recorde. Telas, autenticação, telas de criação, detalhe, listagem, paginação, integração com redis cache, eu fiz também um agente integrado com Plugins do Semantic Kernel que é capaz de ver os exercícios na base e sugerir treinos baseado nas informações que o usuário enviou, validações, testes unitários e integrados, tudo isso em 1 dia. É verdade que o código estava longe da qualidade desejada. A Queda para o Vibecoding O problema de ver tudo funcionando rápido é que o poder sobe à cabeça. A sensação de invencibilidade me fez abandonar o plano original. De repente, eu estava improvisando outras features que não estavam planejadas como gamificação, múltiplos dashboards, admin. O rigor técnico desapareceu. Parei de criticar o código gerado e o resultado foi inevitável: o projeto, antes até que bonitinho, degradou-se rapidamente. O Aprendizado Apesar do caos no código final, houve um brilho técnico: integrei o Playwright com o Gemini via MCP. Ver o modelo "olhando" para a interface e rodando seus próprios testes foi animal. O Spec Kit é uma surpresa, apesar de ser um pouquinho complicado de entender, ele realmente é uma ferramenta que auxilia e da controle na hora de criação das features. As vezes é melhor botar a mão no código e resolver problemas do que escrever mais um prompt e esperar a solução. É preciso saber identificar quando o modelo está patinando no gelo porque a verdade é que quem vai patinar é você e não o modelo. No fim, o Ficha Monstra me ensinou que a IA acelera a construção, mas também acelera a criação de dívida técnica se você baixar a guarda. O código gerado é realmente feio e ruim, e é preciso se tornar mais revisor do que o codificador. E saber as bases da engenharia de software é o que faz a diferença. E me deu uma sensação meio estranha para 2026 de que nada será como antes, MESMO.

dev.to

How I Made a Voice-First Todo List That's Actually Fast (And Why I Rewrote Half of It)

TL;DR: I built a todo app. Then I made it slower by over-engineering it. Then I made it 6x faster by embracing platform capabilities. Here's what I learned about premature optimization, when to actually optimize, and why sometimes the "boring" solution is the right one. The Problem: Why Another Todo App? (Besides the Obvious Answer: "I Have ADHD") Look, I know what you're thinking. "Another todo app? Really? In 2025?" But hear me out. I built WhisperPlan for one simple reason: I hate typing on my phone. Like, really hate it. The autocorrect betrayals, the tiny keyboard, the context switching from "thing I need to do" to "person actively typing on a tiny keyboard." And here's the thing about ADHD (which I have, and which I'm building for): the friction between "I should do this" and "I have written this down" needs to be as close to zero as possible. Because if it takes more than 3 seconds, my brain has already moved on to something else. Probably a Wikipedia deep-dive about Byzantine architecture. So I built WhisperPlan: a voice-first todo list. You press a button, you talk, and boom—tasks appear. Like magic, except it's actually OpenAI's Whisper and Google's Gemini having a conversation about your life. The app includes: Voice-to-tasks in seconds: Record → Transcribe → AI extracts tasks with due dates, priorities, and projects ADHD-friendly gamification: Confetti, celebrations, streaks (because dopamine matters) Focus Mode: One task at a time with a Pomodoro timer (hello, Dynamic Island!) Calendar Integration: Seeing what you have to do this week just by looking at your calendar is magical! This is the story of how I built it, realized it was too slow, tore half of it out, and made it 6x faster. Buckle up. The Stack: Choosing Your Weapons (Or: "The Tech I Already Knew") Let me be honest with you: I didn't choose this stack after careful consideration of all options, extensive benchmarking, and architectural decision records. I chose it because I already knew these technologies and I wanted to ship something, not write a PhD thesis. Here's what I went with: Frontend: iOS (SwiftUI + SwiftData) Why native? Because I wanted widgets, Dynamic Island integration, and that buttery 120fps scroll. Also, my target audience is "people who own iPhones and hate typing," which feels like a pretty specific demographic. SwiftUI is delightful once you stop fighting it. SwiftData is CoreData without the trauma. Together they're like peanut butter and chocolate—if peanut butter occasionally crashed your app because you forgot to mark something @Published. Backend: NestJS on Google Cloud Run Why NestJS? TypeScript comfort zone. Decorators make me feel fancy. Dependency injection makes me feel like a "real" backend developer. Why Cloud Run? Because I wanted serverless-ish (scale to zero, pay for what you use) but I also wanted to deploy a Docker container and not think about it. Plus, the cold start is under 1 second, which is faster than me making decisions in the morning. Database: Cloud Firestore Plot twist: This choice would later save my performance bacon. But initially, I treated it like a boring NoSQL database I had to use because I was already in the Firebase ecosystem. Spoiler alert: Firestore's real-time listeners and offline support are chef's kiss. AI: OpenAI Whisper + Google Gemini Whisper (specifically gpt-4o-transcribe): For transcription. It's scary good. It handles my fast-talking, my accent, and even my tendency to say "um" 47 times per recording. Gemini (gemini-2.5-flash): For extracting structured tasks from transcriptions. Turns "I need to call mom tomorrow at 2pm and also buy milk" into two properly formatted tasks. Auth: Firebase Authentication Sign in with Apple. Because it's 2025 and nobody wants another password. The "boring technology" crowd would approve of most of this. Except maybe the part where I initially over-engineered the backend. We'll get to that. The First Architecture: Everything Through the Backend (AKA "Look Ma, I'm A Real Backend Developer!") Here's how I initially architected WhisperPlan: iOS App → NestJS Backend → Firestore → NestJS Backend → iOS App Every. Single. Operation. Went. Through. The. Backend. Reading tasks? Backend API call. Creating tasks? Backend API call. Updating a task? Backend API call. Marking a checkbox? You guessed it—Backend API call. Why Did This Make Sense At The Time? Look, I had reasons: API keys need to be secret: OpenAI and Gemini keys can't live in the iOS app. ✅ Valid! Quota management for freemium: I needed to enforce "20 tasks/month on free plan" somehow. ✅ Also valid! Backend as source of truth: The backend should control everything, right? That's what we learned in school! ❓ Questionable... I just really like writing TypeScript: ❌ Not a good reason, Isidore. So I built this beautiful, over-engineered sync service. 477 lines of TypeScript that handled: Bidirectional sync (client → server, server → client) Conflict resolution Change tracking Incremental updates A complex state machine Here's a simplified version of what that looked like: // backend/src/sync/sync.controller.ts @Post() async incrementalSync( @CurrentUser() user: CurrentUserData, @Body() dto: IncrementalSyncDto, ) { const results = []; // Process all local changes from the client for (const change of dto.changes) { if (change.action === 'create') { const created = await this.tasksService.create(user.uid, change.data); results.push({ changeId: change.id, serverId: created.id }); } else if (change.action === 'update') { await this.tasksService.update(user.uid, change.serverId, change.data); results.push({ changeId: change.id, action: 'updated' }); } else if (change.action === 'delete') { await this.tasksService.delete(user.uid, change.serverId); results.push({ changeId: change.id, action: 'deleted' }); } } // Fetch all changes from the server since last sync const serverChanges = await this.syncService.getChanges( user.uid, dto.lastSyncAt ); // Check for conflicts (same resource modified in both places) const conflicts = this.syncService.detectConflicts( dto.changes, serverChanges ); return { results, conflicts, serverChanges, syncedAt: new Date().toISOString(), }; } This endpoint did EVERYTHING. It was my baby. I was so proud of it. It was also completely unnecessary. The iOS side was equally complex: // The old BackendSyncService.swift: 477 lines of pain class BackendSyncService: ObservableObject { func syncWithBackend() async throws { // Collect all local changes let changes = try await collectLocalChanges() // Send to backend let response = try await apiClient.post("/sync", body: changes) // Process results for result in response.results { try await applyResult(result) } // Handle conflicts (oh god the conflicts) for conflict in response.conflicts { try await resolveConflict(conflict) // 🔥 This is fine 🔥 } // Apply server changes for change in response.serverChanges { try await applyServerChange(change) } // Update last sync timestamp lastSyncAt = response.syncedAt } } Beautiful, right? Complex, sophisticated, enterprise-grade! Also: slow. Very, very slow. The Wake-Up Call: Performance Metrics (Or: "Why Is My App So Slow?") I launched a beta. People used it. People... waited. A lot. Then I actually measured things (novel concept, I know): App startup: 2-3 seconds Why? Full sync on every cold start User experience? Staring at a loading spinner Task completion: 500-1000ms Why? API call → backend update → response → local update User experience? "Did I tap that? Let me tap it again." Network requests per session: 20-30 requests Why? Because EVERYTHING went through the backend User experience? RIP their cellular data Data transferred: ~100KB per session Why? Full task lists on every sync User experience? Not great, Bob The moment of clarity came when I sat with the app open and actually used it. I completed a task. I waited. The checkbox hung there in limbo. Then—half a second later—it actually checked. Half a second to mark a checkbox. I was making THREE API CALLS just to check a box: PATCH /tasks/:id to update the task GET /sync to fetch updated data GET /tasks to reload the list (just to be safe!) This was ridiculous. This was over-engineering. This was... exactly the kind of thing I make fun of other developers for doing. Time to fix it. The Pivot: Hybrid Architecture (Or: "Wait, Firestore Has An iOS SDK?") Here's the thing about epiphanies: they usually involve realizing you were doing something dumb all along. I was treating Firestore like a dumb database that I had to protect behind an API. But Firestore isn't dumb. Firestore is smart. It has: Real-time listeners (automatic sync) Offline support (with a local cache) Security rules (server-side enforcement) Native SDKs (for iOS, Android, web) I had all of this available and I was... not using it? Because of some vague notion that "backends should control everything"? The Aha Moment I was reading the Firestore documentation (procrastinating, really) and I saw this: "Firestore SDKs include built-in support for offline data persistence. This feature caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline." Wait. WHAT? You mean I can: Read data directly from Firestore (instant, even offline) Update data directly in Firestore (instant, automatic sync) Use security rules to enforce permissions (no need for backend middleware) Let Firestore handle all the real-time sync magic And I've been... wrapping everything in a REST API... for no reason? The New Rules I rewrote the architecture with a simple principle: Use the backend only for things that REQUIRE the backend. Backend ONLY for: ✅ Transcription (needs OpenAI API key) ✅ AI extraction (needs Gemini API key) ✅ Creating tasks (needs quota management) ✅ Deleting tasks (needs quota decrement) ✅ Subscription verification (needs server validation) Firestore Direct Access for: ✅ Reading tasks/projects (real-time listeners) ✅ Updating tasks/projects (instant, offline-first) ✅ Marking tasks complete (no round-trip needed) This is what software architects call a "hybrid approach." I call it "using the right tool for the job." Code Example: Before and After Here's what completing a task looked like before: // BEFORE: The slow way func completeTask(_ task: TaskItem) async throws { // 1. Optimistic update (for perceived speed) task.isCompleted = true // 2. Send to backend let response = try await apiClient.patch("/tasks/\(task.serverId)", [ "isCompleted": true, "completedAt": Date().toISOString() ]) // 3. Wait for response (500ms later...) task.isCompleted = response.isCompleted task.completedAt = response.completedAt // 4. Trigger sync to update other fields that might have changed try await backendSyncService.syncWithBackend() // Total time: 500-1000ms // Network requests: 2-3 } And here's the new way: // AFTER: The fast way func completeTask(_ task: TaskItem) async throws { // 1. Update locally task.isCompleted = true task.completedAt = Date() try modelContext.save() // 2. Update Firestore directly try await firestoreService.updateTask( userId: userId, taskId: task.serverId, updates: [ "isCompleted": true, "completedAt": task.completedAt?.iso8601String ?? NSNull(), "updatedAt": Date().iso8601String ] ) // That's it! Firestore syncs to other devices automatically. // Total time: 50-100ms // Network requests: 1 } 50-100ms. 10x faster. And it works offline! The Security Layer: Firestore Rules Are Magic Now, you might be thinking: "But wait! If clients can write directly to Firestore, what about security? What about quota enforcement?" This is where Firestore security rules save the day. They're basically server-side validators that run on every request: // firestore.rules rules_version = '2'; service cloud.firestore { match /databases/{database}/documents { // Helper function: Check if the user owns this resource function isOwner(userId) { return request.auth != null && request.auth.uid == userId; } // Helper function: Validate task updates function validTaskUpdate() { let allowedFields = ['title', 'description', 'dueDate', 'priority', 'isCompleted', 'completedAt', 'isArchived', 'updatedAt']; return request.resource.data.keys().hasAll(allowedFields) == false; } match /users/{userId}/tasks/{taskId} { // Read: only if you own it allow read: if isOwner(userId); // Update: only if you own it AND the update is valid allow update: if isOwner(userId) && validTaskUpdate(); // Create/Delete: NOPE! Must go through backend allow create, delete: if false; } match /users/{userId}/projects/{projectId} { allow read: if isOwner(userId); allow update: if isOwner(userId); allow create, delete: if false; // Backend only! } } } This is brilliant because: Users can update their own tasks (instant, offline-capable) Users CANNOT create or delete tasks (must go through backend for quota checks) Security is enforced server-side (no client can bypass this) Validation happens automatically (malformed updates are rejected) The backend still handles creates/deletes so I can enforce the freemium quota: // backend/src/tasks/tasks.service.ts async create(userId: string, dto: CreateTaskDto) { // Check quota for free users const user = await this.usersService.findOne(userId); if (user.plan === 'free') { const stats = await this.usersService.getUsageStats(userId); if (stats.activeTasksCount >= 25) { throw new ForbiddenException('Free plan limit reached'); } } // Create task in Firestore const taskRef = await this.firestore .collection(`users/${userId}/tasks`) .add({ ...dto, isCompleted: false, createdAt: FieldValue.serverTimestamp(), updatedAt: FieldValue.serverTimestamp(), }); // Increment counter await this.firestore .doc(`users/${userId}`) .update({ activeTasksCount: FieldValue.increment(1) }); return { id: taskRef.id, ...dto }; } Beautiful. Secure. Fast. The New Sync Service: From 477 Lines to Real-Time Listeners Remember that 477-line sync service? Gone. Replaced with this: // ios/whisperPlan/Services/FirestoreService.swift class FirestoreService: ObservableObject { private var tasksListener: ListenerRegistration? private var projectsListener: ListenerRegistration? func observeTasks(userId: String, modelContext: ModelContext) { tasksListener = db.collection("users/\(userId)/tasks") .addSnapshotListener { snapshot, error in guard let documents = snapshot?.documents else { return } for document in documents { let data = document.data() // Check if task already exists locally let predicate = #Predicate<TaskItem> { $0.serverId == document.documentID } let existing = try? modelContext.fetch( FetchDescriptor(predicate: predicate) ).first if let task = existing { // Update existing task task.title = data["title"] as? String ?? "" task.isCompleted = data["isCompleted"] as? Bool ?? false // ... update other fields } else { // Create new task let task = TaskItem(from: data, serverId: document.documentID) modelContext.insert(task) } } try? modelContext.save() } } func updateTask(userId: String, taskId: String, updates: [String: Any]) async throws { try await db.document("users/\(userId)/tasks/\(taskId)") .updateData(updates) // Done! Firestore will notify all listeners automatically } } That's it. Real-time sync. Offline support. Automatic conflict resolution (last-write-wins based on updatedAt timestamp). All in about 100 lines. The BackendSyncService still exists, but it's now down to 150 lines and only handles: Creating tasks (via backend API) Deleting tasks (via backend API) Creating projects (via backend API) Deleting projects (via backend API) Everything else? Direct Firestore access. The Results: Numbers Don't Lie (Finally, Some Good News!) After rewriting the architecture, I measured again: Metric Before After Improvement App startup 2-3s 0.5s 4-6x faster Task completion 500-1000ms 50-100ms 10x faster Data transferred ~100KB ~5KB 95% less Network requests 20-30 3-5 80% less Cloud Run costs $20-30/mo $5-10/mo 70% savings But the numbers don't tell the whole story. The app feels different now: Offline Mode Just Works™ Because Firestore's SDK has built-in persistence, offline mode is basically automatic: User modifies a task (no internet) Firestore writes to local cache UI updates instantly When internet returns, Firestore syncs automatically Other devices get updates via real-time listeners No complex queue system. No manual retry logic. No sync conflicts to resolve manually. It just works. Real-Time Sync Between Devices The Firestore listeners mean that if I complete a task on my iPhone, it appears as completed on my iPad instantly. No polling. No manual refresh. Magic. // This is all you need for real-time sync: firestoreService.observeTasks(userId: userId, modelContext: modelContext) // Firestore handles: // - Initial data load // - Real-time updates // - Conflict resolution // - Offline caching // - Automatic reconnection // - Everything The first time I saw this work, I literally said "woah" out loud like I was in The Matrix. Lessons Learned for Indie Devs 1. Start Simple, But Not Too Simple My mistake wasn't building a backend—it was building TOO MUCH backend. What I should have done from the start: Use Firestore's native SDKs for CRUD operations Use backend only for things that require secrets or business logic Embrace platform capabilities instead of abstracting them away The trap I fell into: "If I'm building a backend, everything should go through the backend!" No. Just because you have a hammer doesn't mean everything is a nail. Sometimes things are screws. Or maybe they don't need fastening at all and you're just adding complexity. 2. Measure Before Optimizing (But Also Actually Measure) I violated both parts of this: I didn't measure initially (just assumed it was fine) I didn't optimize until it was obviously slow The right approach: Add basic performance monitoring from day 1 Set acceptable targets (e.g., "task completion < 200ms") Measure again after major changes Let data guide your decisions Tools I wish I'd used earlier: Firebase Performance Monitoring (literally free) Xcode Instruments (already installed) Backend latency logging (one line of code) 3. Security Rules Are Your Friend (And Surprisingly Powerful) Firestore security rules are essentially a DSL for server-side validation. They're: Type-safe: Wrong field types are rejected Composable: Functions can call other functions Testable: Firebase Emulator lets you test them locally Fast: Run on Google's infrastructure, not your backend This snippet alone saved me from having to write an entire middleware layer: function validTaskUpdate() { // Only allow specific fields to be updated let allowedFields = ['title', 'description', 'isCompleted', 'completedAt', 'priority', 'dueDate']; let incomingFields = request.resource.data.keys(); return incomingFields.hasAll(allowedFields) == false; } 4. The Freemium Quota Problem (Solved!) Challenge: How do you enforce "25 tasks max" when clients write directly to Firestore? Solution: Split operations by privilege level: Creates/Deletes: Must go through backend (quota enforcement) Reads/Updates: Direct Firestore access (no quota needed) Backend code for enforcement: async create(userId: string, dto: CreateTaskDto) { const user = await this.usersService.findOne(userId); // Check quota if (user.plan === 'free' && user.activeTasksCount >= 25) { throw new ForbiddenException({ error: 'task_limit_reached', message: 'Free plan allows 25 active tasks. Upgrade to Pro for unlimited tasks.', currentCount: user.activeTasksCount, limit: 25 }); } // Create task const taskRef = await this.firestore .collection(`users/${userId}/tasks`) .add(dto); // Increment counter atomically await this.firestore.doc(`users/${userId}`).update({ activeTasksCount: FieldValue.increment(1) }); return { id: taskRef.id }; } This way: Free users can't bypass the limit (creates must use backend) Users get instant updates (no need for backend roundtrips) Counting is atomic (no race conditions) 5. Offline-First Is Easier Than You Think I thought offline support meant: Complex queueing system Manual conflict resolution Edge case nightmares Reality: // Enable persistence (one line) let settings = FirestoreSettings() settings.isPersistenceEnabled = true settings.cacheSizeBytes = FirestoreCacheSizeUnlimited db.settings = settings // That's it. You now have offline support. Firestore handles: Local caching Offline writes (stored locally) Automatic sync when online Conflict resolution (configurable, defaults to last-write-wins) Combined with SwiftData for app-level caching, the user never sees "no connection" errors. They just... use the app. The AI Integration: Voice to Tasks (The Fun Part!) Okay, let's talk about the actual point of WhisperPlan: turning voice into tasks. Step 1: Transcription (OpenAI Whisper) When the user records audio, the iOS app sends it to my backend: // ios/whisperPlan/Services/AudioRecordingService.swift func transcribeRecording() async throws -> String { guard let audioData = try? Data(contentsOf: recordingURL) else { throw RecordingError.fileNotFound } // Send to backend let response = try await apiClient.post("/transcription", multipart: [ "file": audioData, "language": preferredLanguage ] ) return response.text } Backend forwards it to OpenAI: // backend/src/transcription/transcription.service.ts async transcribe(audioBuffer: Buffer, filename: string, language?: string) { const formData = new FormData(); formData.append('file', audioBuffer, { filename }); formData.append('model', 'gpt-4o-transcribe'); // The good stuff formData.append('response_format', 'json'); if (language) { formData.append('language', language); } const response = await axios.post( 'https://api.openai.com/v1/audio/transcriptions', formData, { headers: { 'Authorization': `Bearer ${this.apiKey}`, ...formData.getHeaders() }, timeout: 120000 // 2 minutes for large files } ); return { text: response.data.text, language: response.data.language, duration: response.data.duration }; } The gpt-4o-transcribe model is scary good. It handles: Multiple languages (automatic detection) Accents and speech patterns Background noise Um's, uh's, and filler words (filtered out) Natural pauses and context Step 2: Task Extraction (Google Gemini) Now I have text: "I need to call mom tomorrow at 2pm and also buy milk and don't forget to finish the blog post by Friday" Time to turn that into structured tasks. Enter Google Gemini: // backend/src/tasks/tasks.service.ts (simplified) async extractTasksFromTranscription( userId: string, transcription: string, language: string ) { // Get user's existing projects for context const projects = await this.projectsService.findAll(userId); const projectNames = projects.map(p => p.name); // Build a context-aware prompt with: // - Current date and time (for relative date parsing) // - User's project list (for automatic categorization) // - Language preference (for better understanding) // - Clear JSON schema definition // - Few-shot examples (3-5 examples work best) const prompt = buildTaskExtractionPrompt({ transcription, currentDate: new Date(), projects: projectNames, language, }); const response = await this.geminiService.generate(prompt); const extracted = JSON.parse(response.text); // Validate and sanitize the output return this.validateExtractedTasks(extracted.tasks, projectNames); } Prompt Engineering Lessons I Learned the Hard Way: The quality of task extraction lives or dies by your prompt. Here's what actually matters: 1. Context is King Don't just send the transcription. Send: Current date/time: LLMs need this to parse "tomorrow", "next Friday", "in 2 hours" User's projects: Helps the AI categorize tasks automatically Language: Even if it can auto-detect, being explicit helps Time zone: If your users are global, this matters for "tomorrow at 9am" 2. Structure Your Output Schema Clearly Be extremely specific about the JSON structure you want. I use TypeScript-style definitions right in the prompt: { "tasks": Array<{ title: string; // Max 100 chars description?: string; // Optional, max 500 chars dueDate?: "YYYY-MM-DD"; // ISO format only priority: "low" | "normal" | "high"; // ... etc }> } 3. Few-Shot Examples Are Worth 1000 Words Include 3-5 examples showing: Simple case: "Buy milk" → single task, no date Complex case: "Call John tomorrow at 2pm and email the report by Friday" → two tasks with different due dates Edge cases: Ambiguous priorities, vague timings, multiple projects mentioned 4. Be Explicit About Edge Cases Tell the LLM what to do when: No actionable items exist ("Just thinking out loud...") Dates are ambiguous ("Friday" when it's currently Thursday) Projects don't match existing ones (create new vs. ignore) Priority isn't mentioned (default to "normal") Multiple tasks are crammed into one sentence 5. Validate Everything LLMs hallucinate. Your code should: function validateExtractedTasks(tasks, validProjects) { return tasks .filter(task => task.title && task.title.length > 0) .map(task => ({ ...task, // Clamp priority to valid values priority: ['low', 'normal', 'high'].includes(task.priority) ? task.priority : 'normal', // Validate project exists project: validProjects.includes(task.project) ? task.project : null, // Ensure date is valid dueDate: isValidDate(task.dueDate) ? task.dueDate : null, })); } 6. Iterate Based on Real Usage My first prompt worked 60% of the time. After analyzing 100+ failed extractions, I discovered patterns: Users say "urgent" but mean "high priority" "This week" is ambiguous (does it include today?) British vs American date formats cause confusion Some users dictate entire emails, not just tasks Each discovery led to prompt tweaks and validation rules. The Result: With good prompt engineering and validation, the system now handles: ✅ Multiple tasks in one recording ✅ Relative date parsing ("tomorrow", "next week", "in 3 days") ✅ Time extraction ("at 2pm", "in the morning", "by end of day") ✅ Priority detection (from context like "urgent", "important", "when you have time") ✅ Project categorization (matches against existing projects) ✅ Natural language variations (handles different phrasings of the same intent) Example flow: User says: "Call mom tomorrow at 2pm, buy milk, and finish that blog post by Friday - make it urgent" Transcription: "call mom tomorrow at 2pm and buy milk and finish that blog post by Friday make it urgent" AI extracts: 3 tasks with proper structure ↓ Task 1: "Call mom" - tomorrow, 2pm, normal priority ↓ Task 2: "Buy milk" - no date, normal priority ↓ Task 3: "Finish blog post" - Friday, high priority Validation: Check dates are valid, projects exist, priorities are sane Result: 3 properly structured tasks saved to database Success rate after optimization: ~95% The remaining 5% are usually edge cases like: Very long, rambling recordings with no clear tasks Heavy background noise affecting transcription Extremely vague task descriptions ("do that thing") Uncommon date formats or ambiguous references For these cases, users can manually edit the extracted tasks before saving. The AI pipeline: Voice recording → Whisper transcription → Gemini extraction → Structured tasks in Firestore → Real-time sync to app. The whole process takes 3-5 seconds. Cost Considerations: As an indie dev, API costs matter. Here's what I learned: Whisper (gpt-4o-transcribe): ~$0.006 per minute of audio Tip: Limit recordings to 2 minutes to keep costs predictable Most task lists can be dictated in under 30 seconds Gemini (gemini-2.5-flash): ~$0.00001 per request Super cheap, even with long prompts Flash model is fast enough (200-500ms response time) Total cost per transcription: ~$0.01 on average With 20 free transcriptions/month, that's $0.20 per free user Totally sustainable for a freemium model Is it perfect? No. Does it work 95% of the time? Yes. And that remaining 5% can be edited manually—which is exactly the right tradeoff for a v1 product. The ADHD-Friendly Features (Because Dopamine Matters) Building for ADHD meant focusing on: Reducing friction (voice input, instant feedback) Gamification (dopamine hits for completing tasks) Focus (one thing at a time, hide distractions) Consistency (daily briefings, streaks, reminders) Celebrations & Confetti When you complete a task, you get: ✨ Confetti animation 💬 Encouraging message ("You're crushing it!" or "One down, more to go!") 📳 Haptic feedback (varies by intensity—first task of the day gets a gentle tap, completing everything gets a double-tap) 📊 Streak counter (maintain your momentum!) // Services/CelebrationService.swift func celebrate(for task: TaskItem, context: CelebrationContext) { let intensity = determineIntensity(context) switch intensity { case .light: triggerHaptic(.medium) showMessage("Nice work!") case .medium: triggerHaptic(.success) showConfetti() showMessage("You're on fire! 🔥") case .intense: triggerHaptic(.doubleSuccess) showConfetti(amount: .lots) showMessage("ALL DONE! Take a break, you earned it! 🎉") } } Daily Briefing Notifications Every morning at 8am (customizable), WhisperPlan sends a notification: "Good morning! You have 5 tasks today. Top priority: Finish blog post" Tap the notification → opens daily briefing view with: Weather-appropriate greeting Task count Top 3 priority tasks "Start Focus Mode" button // Services/NotificationService.swift func scheduleDailyBriefing(time: DateComponents) { let content = UNMutableNotificationContent() content.title = "Good morning!" content.body = "You have \(taskCount) tasks today. Top priority: \(topTask.title)" content.sound = .default let trigger = UNCalendarNotificationTrigger( dateMatching: time, repeats: true ) let request = UNNotificationRequest( identifier: "daily-briefing", content: content, trigger: trigger ) UNUserNotificationCenter.current().add(request) } What's Next: The Roadmap WhisperPlan Beta is live on TestFlight now! You can find it at https://testflight.apple.com/join/5XCdyGDr What's next: Better analytics: Time-tracking insights, productivity patterns, weekly/monthly summaries Collaboration: Share projects with others, assign tasks, real-time updates More AI features: Smart scheduling, task priority suggestions, context-aware reminders Apple Watch app: Quick voice recording, timer control, task completion from wrist Siri Shortcuts: "Hey Siri, add task" → voice recording → tasks created Questions for the community: What features would you want in a voice-first todo app? How do you handle task organization? (Projects? Tags? Contexts?) What's your biggest pain point with existing todo apps? Drop your thoughts in the comments! I'm actively building based on feedback. Conclusion: Build, Measure, Rebuild (And That's Okay!) Here's what I learned building WhisperPlan: 1. It's okay to get the architecture wrong the first time. I built a slow, over-engineered backend-heavy app. Then I measured it. Then I fixed it. That's not failure—that's iteration. 2. Use platform capabilities instead of abstracting them away. Firestore has an iOS SDK. SwiftData has offline support. Firebase has real-time listeners. I could have saved myself weeks by using these from the start. 3. Measure everything. I didn't measure initially. That was dumb. When I finally measured, I found obvious problems. Now I measure everything. 4. Security rules are underrated. Firestore security rules let you have the speed of direct database access with the security of backend validation. This is the secret sauce of the hybrid architecture. 5. The joy of indie development: you can rewrite everything. No committees. No architectural review boards. No "but we've always done it this way." Just you, your code, and the freedom to say "this is dumb, let's make it better." 6. Build for yourself first. I built WhisperPlan because I needed it. I have ADHD. I hate typing on my phone. I forget things constantly. Every feature is something I wanted. And that authenticity shows. WhisperPlan is live. It's fast. It's voice-first. It has confetti. Try it out: whisperplan.app And if you're building your own indie app, remember: it's okay to rewrite half of it. Sometimes that's exactly what you need to do. Acknowledgments Thanks to: Everyone who beta tested and gave feedback The SwiftUI community for endless Stack Overflow answers Firebase for building such amazing tools My therapist for helping me manage the ADHD that inspired this app Coffee. So much coffee. If you made it this far, you're amazing. Go build something cool. Or take a nap. Both are valid choices. — Isidore P.S. If you have ADHD and this app sounds useful, try it out! And if you don't have ADHD but you hate typing on your phone, also try it out! And if you're a developer curious about hybrid architectures, I hope this post was helpful! P.P.S. The backend code is not open source, but if you have specific questions about the architecture, hit me up in the comments

dev.to

AWS VPC

If you are new to AWS, VPC (Virtual Private Cloud) is the first and most important networking concept to understand. Every EC2, RDS, Load Balancer, or service you create in AWS lives inside a VPC. Without understanding VPC, AWS networking feels confusing. This blog explains AWS VPC step by step using a simple diagram, so beginners can clearly understand how traffic flows inside AWS. What is AWS VPC? An AWS Virtual Private Cloud (VPC) is a logically isolated network inside AWS where you can launch resources such as: EC2 instances Databases (RDS) Load Balancers You control: IP address range (CIDR) Subnets Routing Security Think of a VPC as your own private data center inside AWS. Why Do We Need a VPC? We need a VPC to: Isolate our AWS resources from others Control inbound and outbound traffic Design secure architectures Decide which resources are public and which are private Without a VPC, you cannot properly secure or control your cloud infrastructure.

dev.to

U.S. conducts strikes on ISIS in Nigeria on Christmas

U.S. conducts strikes on ISIS in Nigeria on Christmas

nbcnews.com

Vercel vs Netlify 2025: The Truth About Edge Computing Performance

The landscape of frontend deployment has undergone a profound transformation in late 2024 and throughout 2025, with Vercel and Netlify leading the charge into a truly distributed, edge-first paradigm. As a developer who's been neck-deep in these platforms, stress-testing their latest features, I can tell you this isn't just marketing hype; the advancements in edge functions and serverless architectures are fundamentally altering how we design and deliver web applications. We're moving beyond mere static site hosting to intelligent, dynamic experiences served closer to the user than ever before. The core shift is from regional serverless functions, which still suffer from network latency for globally distributed users, to lightweight "edge" runtimes that execute code at CDN points of presence (PoPs). This promises not just faster response times but a more resilient and cost-efficient compute model. However, it's not a silver bullet, and understanding the nuances of each platform's approach and the trade-offs involved is paramount. Let me walk you through what's truly changed and how to leverage these powerful tools effectively. Vercel's Edge Runtime: Fluid Compute and the Evolving V8 Isolate Vercel has been systematically refining its serverless offerings, and a significant development in mid-2025 was the unification of "Edge Middleware" and "Edge Functions" under the broader "Vercel Functions" umbrella. This means that what we previously called "Edge Functions" are now "Vercel Functions using the Edge Runtime," and "Edge Middleware" has evolved into "Vercel Routing Middleware." Both now leverage a consistent, unified infrastructure. The underlying technology for Vercel's Edge Runtime remains its strength: a lightweight execution environment built on the V8 JavaScript engine. This isn't a full Node.js environment; instead, it utilizes V8 isolates, providing a minimal API surface that adheres closely to Web Standard APIs like fetch, Request, and Response. This design choice is crucial for its unparalleled cold start performance, which can be up to 9 times faster globally than traditional serverless functions during initial invocation. Even for warm invocations, edge functions are about twice as fast. The isolation boundary ensures secure, multi-tenant execution without the overhead of full virtual machines or containers. Fluid Compute and Active CPU Pricing A groundbreaking announcement at Vercel Ship 2025 was the introduction of Fluid Compute and Active CPU Pricing. Traditionally, serverless functions charged for the entire duration of a request, including idle I/O time. Fluid Compute changes this, allowing you to pay only for the active CPU cycles your function consumes. This is a game-changer for I/O-bound tasks, especially long-running AI inference workloads and streaming APIs, as it drastically reduces costs by not charging for network latency or waiting on external API calls. This cost model significantly enhances the viability of complex, stateful edge applications. Here's exactly how you'd configure a Vercel Function to use the Edge Runtime, specifying a preferred region for optimal data locality: // api/regional-example/route.ts (for Next.js App Router) import { NextRequest, NextResponse } from 'next/server'; export const runtime = 'edge'; // Execute this function on iad1 (US East) or hnd1 (Tokyo), // based on the connecting client's location, // to be closer to a specific database or service if needed. export const preferredRegion = ['iad1', 'hnd1']; export async function GET(request: NextRequest) { // Access Web Standard APIs like Request const userAgent = request.headers.get('user-agent'); console.log(`Request from user agent: ${userAgent}`); // Perform some lightweight computation or external fetch const data = { message: `Hello from Vercel Function (Edge Runtime)!`, region: process.env.VERCEL_REGION, // Vercel injects this timestamp: new Date().toISOString(), }; return new NextResponse(JSON.stringify(data), { status: 200, headers: { 'Content-Type': 'application/json', 'Cache-Control': 's-maxage=1, stale-while-revalidate=59', // Edge caching }, }); } In this example, runtime: 'edge' explicitly opts into the Edge Runtime. The preferredRegion array is critical for scenarios where your Edge Function needs to interact with a regional database or service. While Edge Functions generally run globally closest to the user by default, this allows you to "pin" execution to a region (or a set of regions) that might be geographically closer to your data source, mitigating the "database proximity" problem. Without this, an edge function executing in Europe, but querying a database in the US, would negate some of the latency benefits. Netlify's Edge Functions: Deno's Web-Standard Advantage Netlify's approach to Edge Functions distinguishes itself by embracing Deno as its underlying runtime. This was a deliberate choice, driven by Deno's strong adherence to web standards, built-in TypeScript support, and a security model that makes it well-suited for multi-tenant edge environments. For developers coming from a frontend background, the Deno environment feels familiar, providing standard Web APIs like Request, Response, and URL, rather than Node.js-specific primitives. This is a key differentiator when comparing Cloudflare vs. Deno: The Truth About Edge Computing in 2025, as Deno's strong adherence to web standards simplifies cross-platform logic. Netlify Edge Functions are designed to run at Netlify's network edge, closest to the user, for operations requiring low latency and quick execution (typically under 50 milliseconds). They integrate seamlessly into the Netlify build and deployment workflow, meaning your edge function code is version-controlled, built, and deployed alongside your frontend code. This provides a cohesive developer experience, where the boundary between frontend and backend logic blurs, especially for tasks like request modification, authentication, or personalization. A key feature of Netlify Edge Functions is the context object, which provides access to Netlify-specific capabilities and metadata about the incoming request. This includes geolocation data, cookie management, and powerful methods for rewriting or redirecting requests. This context object is what empowers many of the advanced use cases we'll discuss, such as A/B testing and geo-localization, directly at the edge. Let's look at a basic Netlify Edge Function setup: // netlify/edge-functions/hello-edge.ts import type { Context } from "@netlify/edge-functions"; export default async (request: Request, context: Context) => { // Access request headers const userAgent = request.headers.get("user-agent"); // Access Netlify-specific context, e.g., geolocation const city = context.geo?.city || "unknown"; const country = context.geo?.country?.name || "unknown"; console.log(`Edge Function invoked from ${city}, ${country} by ${userAgent}`); // // Set a cookie (using the context object) context.cookies.set({ name: "edge_visitor", value: "true", expires: new Date(Date.now() + 86400 * 1000).toUTCString(), // 1 day httpOnly: true, secure: true, }); // Return a new response or modify the existing one return new Response(`Hello from Netlify Edge! You are in ${city}, ${country}.`, { headers: { "Content-Type": "text/plain" }, }); }; // netlify.toml - for defining paths and optional configurations [[edge_functions]] function = "hello-edge" path = "/hello-edge" # You can also configure response caching here # cache = "manual" # headers = { "Cache-Control" = "public, max-age=60" } This example demonstrates how to access geolocation data and manage cookies using the context object. The netlify.toml file is used to declare and configure the edge function, providing a clear separation of concerns between code and routing. While inline configuration within the function file (export const config = { path: "/test" }) is also supported, using netlify.toml offers more nuanced control, especially for ordering and advanced settings. Stateful Edge: The Quest for Persistent Data Close to the User One of the long-standing challenges with edge computing has been managing state. Edge functions are inherently stateless, designed for ephemeral execution. However, for truly dynamic and personalized experiences, data persistence at the edge or highly performant access to global data stores is crucial. Both Vercel and Netlify have been making strides in this area. Vercel offers Vercel KV, a Redis-compatible key-value store designed for low-latency data access from Edge and Serverless Functions. While the search results didn't detail recent specific updates to Vercel KV in 2024-2025, its presence is a clear signal of Vercel's commitment to enabling stateful logic at the edge. It's often paired with Edge Config, a low-latency data store for feature flags, A/B test parameters, or dynamic content that needs to be globally available and instantly updated without redeploying functions. Netlify has introduced Netlify Blobs, a solution for storing and retrieving immutable binary data directly from the edge. While the details of its maturity and specific use cases were not extensively covered in the latest search results, its mention in the context of Astro integration suggests it's becoming a viable option for caching or storing content fragments closer to users. Furthermore, Netlify's general approach emphasizes integrations with external, globally distributed databases like PlanetScale or Turso (a SQLite-compatible edge database), which provide the necessary data locality. The performance implications of an Edge Function interacting with a distant database are significant, often negating the edge benefits. This is where solutions like Vercel's preferredRegion for Edge Functions become vital, allowing you to route traffic to a region closer to your data source when necessary. The reality is that truly persistent, mutable data at every edge node is still a complex problem. For most applications, a hybrid approach combining edge functions for request manipulation and a globally distributed, eventually consistent database (or a regional database with careful preferredRegion routing) remains the most practical solution. Edge-Powered Personalization & A/B Testing This is where edge functions truly shine, enabling dynamic experiences without client-side overhead or origin server roundtrips. Both platforms offer robust capabilities for A/B testing and content personalization. Netlify's Edge Functions are exceptionally well-suited for A/B testing. You can intercept a request, assign a user to a test "bucket" based on a random number, set a cookie to remember their assignment, and then rewrite the request or response to serve different content. This happens before the request even reaches your site's origin, eliminating the "flash of unstyled content" (FOUC) or performance degradation often associated with client-side A/B testing tools. Let's outline a practical A/B testing implementation on Netlify: // netlify/edge-functions/ab-test.ts import type { Context } from "@netlify/edge-functions"; export default async (request: Request, context: Context) => { const cookieName = "ab_test_variant"; let variant = context.cookies.get(cookieName); if (!variant) { // If no cookie, assign a variant (e.g., 50/50 split) variant = Math.random() < 0.5 ? "A" : "B"; context.cookies.set({ name: cookieName, value: variant, expires: new Date(Date.now() + 7 * 24 * 60 * 60 * 1000).toUTCString(), // 7 days httpOnly: true, secure: true, path: "/", }); } // Rewrite the request based on the variant // For example, serving different static HTML files or API responses if (variant === "B") { // Rewrite to a different path for variant B content // This could be /index-variant-b.html or /api/data?variant=B return context.rewrite("/variant-b" + request.url.pathname); } // For variant A, let the request proceed as normal (or rewrite to /variant-a) return context.next(); }; // netlify.toml [[edge_functions]] function = "ab-test" path = "/*" # Apply to all paths This setup ensures that a user consistently experiences either variant A or B throughout their session. Netlify's context.rewrite() is incredibly powerful here, allowing you to dynamically change the requested resource at the edge. Vercel also supports A/B testing and personalization, notably through its Edge Middleware (now Vercel Routing Middleware) and the Vercel Edge Config service. Edge Config provides a centralized, low-latency data store for configuration values, feature flags, and experiment parameters. This allows marketers and product managers to update A/B test weights or enable/disable features without requiring a code deployment, with changes propagating globally in milliseconds. When combined with Next.js Middleware, you can perform similar request rewrites and cookie management as shown in the Netlify example. Observability at the Edge: Debugging Distributed Logic Debugging and monitoring distributed systems is notoriously challenging, and edge functions are no exception. With code executing in hundreds of global locations, traditional logging and tracing methods need rethinking. Both Vercel and Netlify have been improving their observability stories. For Vercel, the Vercel Ship 2025 announcements included Enhanced Logging & Tracing with OpenTelemetry support. This is a critical move towards standardized observability, allowing developers to integrate Vercel's telemetry data with existing OpenTelemetry-compatible monitoring solutions. For Edge Runtime functions, you can still use console.log() statements, which appear in the Vercel project logs. However, for a holistic view, integrating with a dedicated observability platform (e.g., DataDog, New Relic, Elastic) via OpenTelemetry is the path forward for complex applications. Netlify offers comprehensive logging for Edge Functions, displaying console statements with up to 7 days of retention (depending on your plan). More importantly, for Enterprise plans, Netlify provides a Log Drains feature. This allows you to stream site traffic logs, function logs, and edge function logs to third-party monitoring services like Datadog, New Relic, Axiom, Azure Monitor, Sumo Logic, Splunk Observability Cloud, or even Amazon S3. This is invaluable for deep analysis, custom alerting, and long-term data persistence. Here's how you might configure a Netlify Log Drain in the UI (Enterprise feature): Navigate to your site in the Netlify UI. Go to Logs & Metrics > Log Drains. Select Enable a log drain. Choose your external monitoring provider (e.g., Datadog). Select the Log types to drain, ensuring "edge function log output" is checked. Configure service-specific settings (e.g., API key, region). For practical debugging, always start with local emulation using the respective CLIs (vercel dev or netlify dev). Both provide a local environment that closely mimics the production edge runtime, including access to environment variables and context objects. When issues arise in production, correlate your function logs with CDN access logs and any external monitoring data. The distributed nature means an issue might be regional, so look for patterns across different PoPs. Performance Deep Dive: Cold Starts, Latency, and Workload Matching The defining characteristic of edge functions is their performance, primarily driven by reduced latency and faster cold starts compared to traditional serverless functions. Cold Starts: Edge functions generally exhibit significantly lower cold start times. On Vercel, Edge Functions are approximately 9 times faster during cold starts globally compared to Serverless Functions. Netlify's Deno-based Edge Functions are also noted for their much quicker cold start times compared to equivalent Node.js serverless applications. This is due to the lightweight V8 or Deno runtimes and the efficient allocation mechanisms at the edge. While cold starts are still a factor (a delay of 50ms - 1500ms for infrequently used functions), they affect less than 1% of requests for frequently accessed ones. Latency: By executing code closest to the user, edge functions drastically reduce network latency. This is particularly beneficial for global audiences. A request from Arizona to a local edge node will be significantly faster than one routed to a centralized server in London. This global distribution is automatic; Vercel deploys Edge Runtime functions globally, executing them in the PoP closest to the incoming request. Workload Matching: Despite the performance benefits, edge functions are not a universal solution. They are best suited for: Short, performance-critical operations: Request rewrites, header manipulation, authentication checks, A/B testing, geo-localization, and lightweight API responses. I/O-bound tasks: With Vercel's Fluid Compute, long-running I/O operations (like fetching from external APIs) become more cost-effective. However, edge functions have limitations: Restricted Runtimes: They typically lack full Node.js API access (e.g., no file system access, limited native modules). This means complex backend logic, heavy computation, or operations requiring specific Node.js modules are better suited for traditional serverless functions (e.g., Vercel Functions with Node.js runtime, Netlify Functions based on AWS Lambda). Execution Duration: Vercel Edge Functions must begin sending a response within 25 seconds and can stream for up to 300 seconds. Netlify Edge Functions have an execution limit of 50 milliseconds, making them ideal for very short, early-in-the-request-chain operations. Serverless functions, in contrast, can run for much longer (up to 10 seconds or 15 minutes for Netlify Background Functions). The choice often boils down to a hybrid model: use edge functions for the initial, user-facing, high-performance logic, and traditional serverless functions for heavier, longer-running backend processes that might interact with regional databases. Deployment & Developer Experience: CLI, Git, and Local Emulation Both Vercel and Netlify excel in providing a seamless developer experience, deeply integrating with Git and offering powerful CLIs for local development and direct deployments. Vercel's Deployment Workflow: Vercel's Git integration is highly optimized, automatically triggering deployments on every commit or pull request. For local development, the Vercel CLI is indispensable: # Install Vercel CLI globally npm i -g vercel # In your project root, start a local development server vercel dev vercel dev emulates the Vercel environment locally, including Edge Functions (now Vercel Functions using the Edge Runtime) and API routes. For production deployments, you can push to Git or use the CLI directly: # Deploy to a preview environment vercel # Deploy directly to production vercel --prod Vercel's platform also provides features like Deploy Hooks, allowing external systems to trigger deployments, and a robust REST API for programmatic deployment. The integration with frameworks like Next.js (especially the App Router) is first-class, with automatic configuration and optimized bundling. Netlify's Deployment Workflow: Netlify also offers a tightly integrated Git-based workflow, with atomic deploys, deploy previews for every pull request, and instant rollbacks. The Netlify CLI provides excellent local development and deployment capabilities for Edge Functions: # Install Netlify CLI globally npm install -g netlify-cli # In your project root, start a local development server netlify dev netlify dev automatically detects and runs your Netlify Edge Functions locally, even installing Deno if it's not already present on your system. This local emulation is crucial for rapid iteration. For deploying to production: # Log in to Netlify (if not already) netlify login # Deploy your site (including Edge Functions) netlify deploy --prod --build Netlify's adapter for Astro, for instance, automatically compiles Astro middleware into Netlify Edge Functions, enabling SSR at the edge and providing access to the context object via Astro.locals.netlify.context. This framework-agnostic yet deeply integrated approach simplifies the developer's life significantly. The Road Ahead: Unresolved Challenges and Emerging Patterns While edge computing has matured significantly in 2024-2025, there are still areas where the developer experience can be clunky or where fundamental challenges persist. Unresolved Challenges Complex Stateful Logic: While Vercel KV and Netlify Blobs address some storage needs, managing highly mutable, globally consistent, and complex state across many edge locations without introducing significant latency or consistency issues remains a hard problem. Many solutions still involve a centralized database as the source of truth, requiring careful architectural design to minimize edge-to-origin roundtrips. Vendor Lock-in Concerns: Both platforms offer proprietary extensions and contexts (e.g., Netlify's context object, Vercel's preferredRegion). While they build on open runtimes (V8, Deno) and web standards, leveraging their advanced features inevitably ties you closer to their ecosystems. Vercel, however, has recently committed to an "Open SDK" strategy, aiming for loose coupling and portability of their tools across platforms, which is a welcome development. Bundle Size Limits: Edge runtimes are lean, and while limits have increased for Enterprise/Pro teams, developers still need to be mindful of function bundle sizes. This encourages modularity and careful dependency management. Emerging Patterns Hybrid Architectures as the Standard: The future isn't purely "edge" or "serverless," but a thoughtful combination. Edge for initial request handling, authentication, personalization, and caching; serverless for background jobs, database writes, and heavy computation. AI at the Edge: Vercel Ship 2025 highlighted AI integration as a major focus. Edge-optimized AI responses, AI Gateway for seamless LLM switching, and the Vercel AI SDK are pushing AI inference closer to the user, reducing latency for real-time AI applications. This is a fertile ground for new development, where the low latency of the edge can significantly improve the user experience of AI-powered features. WebAssembly (Wasm) at the Edge: Both Vercel and Netlify Edge Functions support WebAssembly, allowing developers to run code written in languages like Rust or C/C++ directly at the edge. This is a powerful enabler for computationally intensive tasks or porting existing high-performance libraries to the edge, potentially overcoming some of the runtime limitations of JavaScript/TypeScript. In conclusion, the advancements from Vercel and Netlify in 2024-2025 have solidified edge functions as a critical component of modern web architecture. With faster cold starts, lower latency, and powerful customization capabilities, they empower developers to build incredibly performant and personalized experiences. However, it's essential to understand their limitations and strategically combine them with traditional serverless functions and robust data solutions to build truly resilient and scalable applications. The "expert colleague" advice here is: test, benchmark, and choose the right tool for the job – often, that means a symphony of edge and serverless working in concert. Sources vercel.com upstash.com vercel.com vercel.com medium.com 🛠️ Related Tools Explore these DataFormatHub tools related to this topic: JSON Formatter - Format vercel.json configs Sitemap Builder - Generate sitemaps for deployment 📚 You Might Also Like AWS re:Invent 2025 Deep Dive: The Truth About Lambda and S3 Cloudflare vs. Deno: The Truth About Edge Computing in 2025 Serverless PostgreSQL 2025: The Truth About Supabase, Neon, and PlanetScale This article was originally published on DataFormatHub, your go-to resource for data format and developer tools insights.

dev.to

You're Telling Me I Get Paid?

This hypothetical keeps circulating on social media like some kind of impossible challenge. A month in a cabin with food, water, and firewood but no internet, phone, or TV. The reward: $100,000. I need everyone to understand something. I would do this for free. You're offering to pay me? I've spent two decades in tech. I've answered "quick questions" at 11pm. I've debugged production issues on Christmas morning. I've felt my soul leave my body during the fourth video call of the day where someone shares their screen to show me an email they could have just forwarded. My nervous system has been marinated in Teams notifications for so long that I twitch when I hear any sound vaguely resembling a ping. A month of enforced silence sounds less like a challenge and more like a prescription. The comments on these posts always fascinate me. "I could never do it." "I'd go crazy." And I think... would you? Or have we just forgotten what our brains feel like without the constant dopamine drip of notifications? I'm genuinely not sure I remember what boredom feels like. Real boredom. The kind that eventually transforms into creativity or rest or just staring at trees like some kind of functioning mammal. So yes. I would absolutely do this. I'd walk into that cabin like I was checking into a spa. The only real question is whether they'd have to drag me out at the end.

dev.to

VPC Part 2 : AWS Site-to-Site VPN (On-Prem Simulation)

Connecting an "on-premise" network to an AWS VPC is the most common real-world enterprise scenario.However, you cannot use VPC Peering for this. In the real world,you use AWS Site-to-Site VPN or AWS Direct Connect. Peering is for AWS-to-AWS only. VPN is for Anything-to-AWS. You use it to connect your home office, a physical data center, or even a different cloud provider (like Azure or Google Cloud) to your AWS VPC. The Concept VPC-A (The Cloud): Uses an AWS Virtual Private Gateway (VGW). VPC-B (The On-Premise): Uses an EC2 instance running strongSwan (an open-source VPN software) to act as your "Corporate Firewall/Router." This is called a Customer Gateway (CGW). Using static routing or BGP (Border Gateway Protocol), you create a secure IPsec VPN tunnel between the two gateways over the public internet. 1. Cost Analysis (Still < $2) Site-to-Site VPN Connection: ~$0.05 per hour. EC2 (t3.micro): ~$0.01 per hour. Public IP: ~$0.005 per hour. Total: If you run this for 2 hours, it will cost roughly $0.15 - $0.20. 2. The Terraform Strategy To simulate this, we need to: Create VPC-A (Cloud) and VPC-B (On-Prem). In VPC-A, create a Virtual Private Gateway (VGW). In VPC-B, create an EC2 instance. This instance needs an Elastic IP. Tell AWS that the "Customer Gateway" is the Public IP of that EC2 instance. Create the VPN Connection. Update Route Tables in both VPCs to allow traffic flow. Route Propagation: Unlike Peering, where you manually add routes, in a VPN setup you can enable "Route Propagation." This allows the VGW to automatically tell the VPC about the on-prem routes it learns via BGP (Border Gateway Protocol). Encryption (IPsec): This project teaches you that traffic between on-prem and AWS is encrypted in transit over the public internet, unlike Peering which stays on the private AWS backbone. The "Customer Gateway" Concept: You learn that AWS doesn't "reach out" to on-prem; you have to define the entry point (CGW) and establish a tunnel. Security Groups / NACLs: You will have to allow UDP Port 500 and UDP Port 4500 (ISAKMP/IPsec) for the tunnel to even start. Project Agenda: Apply Terraform: Run terraform apply. Get the Config: Go to AWS Console > VPC > Site-to-Site VPN Connections. Select your new connection and click Download Configuration.Select Vendor: Generic. Find the Tunnel Info: Open the text file. Look for Tunnel 1. You will see an Outside IP Address (AWS side) and a Pre-Shared Key. Prepare the Bash Script: sample bash script(refer bash.sh file) to configure strongSwan on your On-Prem EC2 instance. Replace the placeholders with the actual values from the text file. Run Bash Script: SSH into your "OnPrem-Router" EC2. Paste the bash script above, replacing the placeholders with the data from the text file. Check Status: In the AWS Console, the VPN Tunnel 1 status should change from Down to Up (Green) after about 1-2 minutes. Test connection. If it works, congratulations! You have successfully connected your on-prem network to your AWS VPC using Site-to-Site VPN. Cost Management Checklist VPN Connection: $0.05/hour.(Deleted immediately after testing). EC2 t3.micro: $0.01/hour. Public IP: $0.005/hour per IP. Total: If you finish this in 2 hours, you will spend roughly $0.25. Steps to Run the Project and make VPN UP Note: Amazon Linux 2023 (AL2023) uses dnf and recommends Libreswan instead of strongSwan. Once you can finally connect to your instance, run this script. Download Config : Go to AWS Console > VPN > Site-to-Site VPN > Download Configuration. Select Vendor: Generic. Get Tunnel 1 Data: Find the Pre-Shared Key and the Virtual Private Gateway IP (called "Outside IP"). Run this on the On-Prem EC2 : Install VPN software sudo dnf install libreswan -y Enable IP Forwarding (Essential for a Router) sudo sysctl -w net.ipv4.ip_forward=1 echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.conf Create the Secrets file (Replace placeholders) ##Format: <OnPrem_Public_IP> <AWS_VPN_Outside_IP> : PSK "<Your_Pre_Shared_Key>" sudo vi /etc/ipsec.d/aws.secrets Create the Tunnel config sudo vi /etc/ipsec.d/aws.conf Paste this into aws.conf (Replace the bracketed IPs): conn tunnel1 authby=secret auto=start left=%defaultroute leftid=[YOUR_ONPREM_EIP_PUBLIC_IP] leftsubnet=192.168.0.0/16 right=[AWS_TUNNEL_OUTSIDE_IP] rightsubnet=10.10.0.0/16 ike=aes128-sha1;modp2048 phase2alg=aes128-sha1;modp2048 keyexchange=ike ikev2=no type=tunnel run the following commands to restart strongSwan and bring up the tunnel: sudo systemctl restart ipsec sudo ipsec auto --add tunnel1 sudo ipsec auto --up tunnel1 Finally, start the service: sudo systemctl start ipsec sudo systemctl enable ipsec sudo ipsec status # Check if "tunnel1" is loaded Testing From your On-Prem EC2, try to ping a private EC2 in the Cloud VPC (10.10.x.x). You should see replies if everything is set up correctly! Learning Outcome : You will finally understand how packets know where to go when they leave a private subnet. You'll see how the "Virtual Private Gateway" handles the cloud side and how a "Customer Gateway" handles the on-prem side.

dev.to

The 10 best shows to stream on Amazon Prime Video from 2025

Aside from free shipping, Prime Video is the second-best perk that comes with an Amazon Prime subscription. This year, the streaming service delivered several strong installments to some of its biggest series like Reacher, Fallout, and Invincible - but it also premiered some great new shows and spinoffs to keep things fresh. Here's a list [&#8230;]

theverge.com

Perspectives on Networking

When you’re new to networking, your perspective is usually very simple — you use the network. You connect your laptop, open a browser, and expect the internet to behave itself. A network engineer, on the other hand, looks at the same setup and sees cables, devices, protocols, and potential failure points just waiting for the wrong moment to strike. Neither perspective is wrong. They’re just different viewpoints of the same system. Let’s start with the most common one: how everyday users experience networking. Typical Network Usage In a basic home setup, a PC connects to a cable modem using an Ethernet cable. That cable modem then connects to a cable TV outlet, also known as a CATV wall outlet, to access the internet. If that sentence sounded like technical noise, don’t worry — that’s normal. Let’s slow it down. Think of the internet as a massive highway. Your PC is a car that wants to get on that highway. Unfortunately, your PC can’t just merge directly into internet traffic like a reckless driver. It needs a gatekeeper. That gatekeeper is the cable modem. The cable modem connects your home network to your ISP, or Internet Service Provider. The ISP owns the highway and decides who gets access. Data from your ISP travels through the cable TV line, reaches your modem, and the modem translates it into something your PC can actually understand. Without this translation step, your device would just be staring at raw signals wondering what it did wrong. Now let’s look at a slightly different setup. Instead of a PC connected by Ethernet, imagine a tablet using Wi-Fi, also called a Wireless Local Area Network (WLAN). No cables, no plugging things in — just vibes and radio waves. In many setups like this, the internet connection comes through DSL, which stands for Digital Subscriber Line. DSL uses your telephone line to deliver internet access. Yes, the same line that was once used only for phone calls now carries internet data as well. To make that possible, a DSL modem connects to the phone line and separates voice signals from data signals, converting the internet traffic into a form your wireless devices can use. So while it looks like magic from the outside, there’s still a lot of networking discipline happening behind the scenes to keep things smooth. Enterprise Networks When networking moves beyond homes and into organizations, things scale up quickly. A network built and maintained by a corporation to allow its employees, systems, and services to communicate is called an enterprise network. These networks are designed for reliability, performance, and security. Downtime isn’t just annoying here — it’s expensive. Enterprise networks support hundreds or thousands of devices, multiple locations, and strict access controls. This is where networking stops being “plug and play” and starts becoming a serious engineering discipline. SOHO Networks Somewhere between a home network and a full enterprise setup lives the SOHO network, short for Small Office / Home Office network. These are typically home networks used for business purposes — freelancers, small startups, or remote workers running professional operations from home. They’re smaller than enterprise networks but often more complex than a typical household setup. SOHO networks still need to be reliable and secure, but without the massive infrastructure of a corporate environment. Think of them as networking’s middle child — doing important work, just on a smaller scale.

dev.to

SwiftUI vs. React Native

I have been working with SwiftUI and React Native for few years and it has been eye opening to see that both declarative UI frameworks have a lot in common. This becomes obvious when implementing custom views.Here's a simple example of a LocationCardView: SwiftUI struct LocationCardView: View { @Environment(Theme.self) private var theme var locationCoordinate: LocationCoordinate var body: some View { VStack(alignment: .leading, spacing: theme.spacing.md) { Text(locationCoordinate.city) .font(.headline) Text(locationCoordinate.state) .font(.caption) Text(locationCoordinate.countryCode) .font(.caption) } .locationCardViewStyle(theme) } } React Native const LocationCardView = (props: LocationCardViewProps) => { const {location} = props const { theme } = useThemeManager((state) => state) const styles = createLocationCardViewStyles(theme, props) return ( <View style={styles.container}> <Text style={[theme.typography.h6Headline]}>{location.city}</Text> <Text style={[theme.typography.caption]}>{location.state}</Text> <Text style={[theme.typography.caption]}>{location.countryCode}</Text> </View> ) } LocationCardView in Simulator As you can see, it takes similar number of lines to accomplish the task.But SwiftUI provides basic primitives like the VStack out of the box.This reduces memory overload and time. And personally, I prefer the DX of SwiftUI. What do the think about both frameworks and which do you prefer? Let us know in the comments.

dev.to

LG teases a new chore-completing home robot

LG is getting ready to take the wraps off a new robot it claims is capable of performing a "wide range" of household chores. The robot, called LG CLOiD, will make its debut at CES next month, featuring two articulated arms and five individually actuated fingers on each hand. Though we only have a description [&#8230;]

theverge.com

ReCode: The AI-Powered Workspace for Web Development

This is a submission for the DEV's Worldwide Show and Tell Challenge Presented by Mux What I Built ReCode is an AI-powered productivity hub designed specifically for web developers or any other person interested. It solves the "tab-overload" problem by centralizing essential development tools into one organized interface, helping developers stay focused and write better code faster. My Pitch Video 🎥 Click here to watch the demo on Mux Demo Live Demo: recode-alpha.vercel.app GitHub Repo: Georgel0/ReCode The Story Behind It As a beginner in web development, I often found myself overwhelmed. I would have dozens of tabs open—documentation, Stack Overflow, AI chats, and tutorials—leading to unorganized code and a fragmented workflow. I built ReCode to be the "all-in-one" assistant I wish I had when I started. By bringing these tools together, ReCode helps developers transition from being "scattered" to being "structured." Technical Highlights To ensure the app was fast, scalable, and modern, I chose a powerful tech stack: Frontend: React + Vite for a lightning-fast user interface and modular component architecture. Backend: Node.js for efficient server-side logic. Database & Auth: Firebase for real-time data handling and secure anonymous user authentication. AI Engine: Grok API to power the intelligent coding assistance features. Video Hosting: Powered by Mux, ensuring high-quality, seamless video delivery for the platform's demo. I designed the architecture to be highly modular, making it easy to implement new features and maintainable for future growth. About the Developer I am a solo developer on a mission to make ReCode the ultimate tool for developers at all levels—from beginners taking their first steps to pros looking for a more streamlined workflow.

dev.to

Report on Artificial Intelligence Developments

Full text is unavailable in the news API lite version

foxnews.com

Built a simple Slack notification

Built a simple n8n workflow that sends Tally form submissions directly to Slack. The goal was to remove manual checks and make sure teams get notified instantly. Simple setup, but very effective for internal workflows.

dev.to

AWS CDK 100 Drill Exercises #005: CDK Parameters —— Managing Parameters with TypeScript vs cdk.json

Introduction This is the fifth installment of "AWS CDK 100 Drill Exercises." For more information about AWS CDK 100 Drill Exercises, please refer to this article. When developing CDK applications, you'll face the challenge of how to manage different configuration values for each environment. Small instances for development, large instances for production. One NAT Gateway for development, multiple across AZs for production. How should you manage these environment-specific configurations? In this exercise, we'll implement two main approaches for managing parameters in CDK and understand the pros and cons of each. Why is Parameter Management Important? Environment Isolation: Use different configurations for development, staging, and production Code Reusability: Reuse the same code with different configurations Configuration Visibility: Make it clear what values are being used Ease of Change: Change configurations without modifying code Type Safety: Leverage TypeScript's type system for safe configuration management What You'll Learn How to define parameters in TypeScript files (type-safety focused) How to define parameters in cdk.json (flexibility focused) Environment-specific parameter management and best practices Parameter validation at deployment time Pros and cons of each approach When to use which approach in production 📁 Code Repository: All code examples for this exercise are available on GitHub. Architecture Overview Here's what we'll build: We'll implement two parameter management approaches. 1. TypeScript Parameters (CdkTSParametersStack) Define parameters in TypeScript files Type safety and IDE support Separate parameter files for each environment Compile-time validation 2. cdk.json Parameters (CdkJsonParametersStack) Define parameters in the context section of cdk.json JSON-based flexible configuration Standard CDK approach Easy dynamic value retrieval Both approaches create the same VPC resources, but differ in how parameters are defined and type safety is ensured. Prerequisites To proceed with this exercise, you'll need: AWS CLI v2 installed and configured Node.js 20+ AWS CDK CLI (npm install -g aws-cdk) Basic knowledge of TypeScript AWS Account (can be done within free tier) Basic understanding of VPC concepts (refer to Episode 3: VPC Basics) Project Directory Structure cdk-parameters/ ├── bin/ │ └── cdk-parameters.ts # Application entry point ├── lib/ │ ├── stacks/ │ │ ├── cdk-ts-parameters-stack.ts # TypeScript parameters stack │ │ └── cdk-json-parameters-stack.ts # cdk.json parameters stack │ └── types/ │ ├── common.ts # Common type definitions │ ├── vpc.ts # VPC type definitions │ └── index.ts # Type definitions export ├── parameters/ │ ├── environments.ts # Environment definitions and parameter interfaces │ ├── dev-params.ts # Development environment parameters │ ├── stg-params.ts # Staging environment parameters │ ├── prd-params.ts # Production environment parameters │ └── index.ts # Parameter exports ├── test/ │ ├── compliance/ │ │ └── cdk-nag.test.ts # Compliance tests │ ├── snapshot/ │ │ └── snapshot.test.ts # Snapshot tests │ └── unit/ │ ├── cdk-ts-parameters-stack.test.ts # Unit tests │ └── cdk-json-parameters-stack.test.ts ├── cdk.json # CDK configuration and JSON parameters ├── package.json └── tsconfig.json Pattern 1: Defining Parameters in TypeScript Files Creating Type Definitions First, define the types for parameters. This enables IDE autocomplete and compile-time type checking. // lib/types/index.ts import * as ec2 from 'aws-cdk-lib/aws-ec2'; export enum Environment { DEVELOPMENT = 'dev', STAGING = 'stg', PRODUCTION = 'prd', TEST = 'test', } export interface SubnetConfig { subnetType: ec2.SubnetType; name: string; cidrMask: number; } export interface VpcCreateConfig { vpcName?: string; cidr: string; maxAzs?: number; natCount?: number; enableDnsHostnames?: boolean; enableDnsSupport?: boolean; subnets?: SubnetConfig[]; } export interface VpcConfig { existingVpcId?: string; createConfig?: VpcCreateConfig; } Key Points: Detailed type definitions like SubnetConfig and VpcCreateConfig prevent configuration mistakes Using enum for environment names prevents typos Optional properties (?) allow for default value usage Environment-Specific Parameter Files Create parameter files for each environment. // parameters/environments.ts export interface EnvParams { accountId?: string; vpcConfig: VpcConfig; } export const params: Partial<Record<Environment, EnvParams>> = {}; // parameters/dev-params.ts import * as ec2 from 'aws-cdk-lib/aws-ec2'; import * as types from 'lib/types'; import { params, EnvParams } from 'parameters/environments'; const devParams: EnvParams = { accountId: '111122223333', // Development environment AWS Account ID vpcConfig: { createConfig: { vpcName: 'DevVPC', cidr: '10.10.0.0/16', maxAzs: 2, // Only 2 AZs for development natCount: 1, // Only 1 NAT for cost savings enableDnsHostnames: true, enableDnsSupport: true, subnets: [ { subnetType: ec2.SubnetType.PUBLIC, name: 'Public', cidrMask: 24, }, { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, name: 'Private', cidrMask: 24, }, ], }, }, }; // Register in global params object params[types.Environment.DEVELOPMENT] = devParams; // parameters/prd-params.ts import * as ec2 from 'aws-cdk-lib/aws-ec2'; import * as types from 'lib/types'; import { params, EnvParams } from 'parameters/environments'; const prdParams: EnvParams = { accountId: '999988887777', // Production environment AWS Account ID vpcConfig: { createConfig: { vpcName: 'PrdVPC', cidr: '10.0.0.0/16', maxAzs: 3, // 3 AZs for redundancy in production natCount: 3, // NAT Gateway in each AZ enableDnsHostnames: true, enableDnsSupport: true, subnets: [ { subnetType: ec2.SubnetType.PUBLIC, name: 'Public', cidrMask: 24, }, { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, name: 'Private', cidrMask: 24, }, ], }, }, }; params[types.Environment.PRODUCTION] = prdParams; Key Points: Different configurations for development and production (number of AZs, NAT Gateways) Including account ID prevents deployment to wrong accounts Type safety detects configuration errors at compile time Stack Using Parameters // lib/stacks/cdk-parameters-stack.ts import * as cdk from 'aws-cdk-lib/core'; import { Construct } from 'constructs'; import { VpcConfig, Environment } from 'lib/types'; import * as ec2 from 'aws-cdk-lib/aws-ec2'; import { pascalCase } from 'change-case-commonjs'; export interface StackProps extends cdk.StackProps { project: string; environment: Environment; isAutoDeleteObject: boolean; vpcConfig: VpcConfig; } export class CdkTSParametersStack extends cdk.Stack { public readonly vpc: ec2.IVpc; constructor(scope: Construct, id: string, props: StackProps) { super(scope, id, props); // Reference existing VPC if (props.vpcConfig.existingVpcId) { this.vpc = ec2.Vpc.fromLookup(this, 'VPC', { vpcId: props.vpcConfig.existingVpcId, }); return; } // Create new VPC if (props.vpcConfig.createConfig) { const createConfig = props.vpcConfig.createConfig; const vpcNameSuffix = createConfig.vpcName ?? 'vpc'; this.vpc = new ec2.Vpc(this, 'VPC', { vpcName: `${pascalCase(props.project)}/${pascalCase(props.environment)}/${pascalCase(vpcNameSuffix)}`, ipAddresses: ec2.IpAddresses.cidr(createConfig.cidr), maxAzs: createConfig.maxAzs || cdk.Stack.of(this).availabilityZones.length, natGateways: createConfig.natCount || 1, subnetConfiguration: createConfig.subnets || [ // Default subnet configuration { subnetType: ec2.SubnetType.PUBLIC, name: 'Public', cidrMask: 24, }, { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, name: 'Private', cidrMask: 24, }, ], enableDnsHostnames: createConfig.enableDnsHostnames ?? true, enableDnsSupport: createConfig.enableDnsSupport ?? true, }); } else { throw new Error('VPC configuration is required to create the VPC.'); } } } Key Points: Type-safe parameter passing through VpcConfig interface Supports both existing VPC reference and new creation Default values allow operation with minimal parameters Entry Point and Deployment Validation // bin/cdk-parameters.ts #!/usr/bin/env node import * as cdk from 'aws-cdk-lib/core'; import { pascalCase } from "change-case-commonjs"; import { params } from "parameters/environments"; import { CdkParametersStage } from 'lib/stages/cdk-parameters-stage'; import { Environment } from 'lib/types/common'; import { validateDeployment } from '@common/helpers/validate-deployment'; import 'parameters'; const app = new cdk.App(); const pjName: string = app.node.tryGetContext("project"); const envName: Environment = app.node.tryGetContext("env") || Environment.DEVELOPMENT; const defaultEnv = { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION, }; // Check for parameter existence if (!params[envName]) { throw new Error(`No parameters found for environment: ${envName}`); } // Pre-deployment validation validateDeployment(pjName, envName, params[envName].accountId); const isAutoDeleteObject = true; const isTerminationProtection = false; new CdkParametersStage(app, `${pascalCase(envName)}`, { project: pjName, environment: envName, env: defaultEnv, terminationProtection: isTerminationProtection, isAutoDeleteObject: isAutoDeleteObject, params: params[envName], }); cdk.Tags.of(app).add("Project", pjName); cdk.Tags.of(app).add("Environment", envName); Deployment Validation Implementation: // common/helpers/validate-deployment.ts export function validateDeployment( pjName: string, envName: string, accountId?: string ): void { console.log(`Project Name: ${pjName}`); console.log(`Environment Name: ${envName}`); // Account ID validation if (accountId) { const isSameAccount = accountId === process.env.CDK_DEFAULT_ACCOUNT; if (!isSameAccount) { const warningBox = [ '', '╭────────────────────────────────────────────────────────────╮', '│ ❌ ACCOUNT MISMATCH WARNING │', '│ │', '│ The provided account ID does not match the current │', '│ CDK account. │', '│ │', `│ Expected: ${accountId} │`, `│ Current: ${process.env.CDK_DEFAULT_ACCOUNT} │`, '│ │', '╰────────────────────────────────────────────────────────────╯', '', ].join('\n'); console.log(warningBox); throw new Error('Account ID mismatch. Deployment aborted.'); } } // Production environment deployment confirmation if (envName === 'prd') { const cautionBox = [ '', '╭────────────────────────────────────────────────────────────╮', '│ 🚨 PRODUCTION DEPLOYMENT │', '│ │', '│ This is a production release. │', '│ Please review carefully before proceeding. │', '│ │', '╰────────────────────────────────────────────────────────────╯', '', ].join('\n'); console.log(cautionBox); const readlineSync = require('readline-sync'); const answer = readlineSync.question( 'Are you sure you want to proceed? (yes/no): ' ); if (answer.toLowerCase() !== 'yes') { throw new Error('Deployment aborted by user.'); } console.log('✓ Proceeding with deployment...'); } } Key Points: Account ID validation prevents deployment to wrong accounts Requires user confirmation before production deployment Visually clear box display In the actual implementation, ANSI color codes are used to customize the display (red for errors, yellow for warnings, green for success messages) for better visibility in the terminal Deployment Method # Deploy to development environment npm run stage:deploy:all --project=myproject --env=dev # Deploy to production environment (with confirmation prompt) npm run stage:deploy:all --project=myproject --env=prd Pros and Cons of TypeScript Approach Pros: Type Safety: Detect configuration errors at compile time IDE Support: Autocomplete and refactoring Complex Logic: Easy parameter calculation and conditional branching Reusability: Share common type definitions across multiple stacks Version Control: Track parameter changes with Git Cons: Recompilation on Changes: Build required every time parameters change Initial Setup: Type definitions and file structure preparation needed Learning Cost: TypeScript knowledge required Pattern 2: Defining Parameters in cdk.json Parameter Definition in cdk.json // cdk.json { "app": "npx ts-node --prefer-ts-exts bin/cdk-parameters.ts", "context": { "dev": { "vpcConfig": { "createConfig": { "vpcName": "DevVPC", "cidr": "10.100.0.0/16", "maxAzs": 2, "natCount": 1, "enableDnsHostnames": true, "enableDnsSupport": true, "subnets": [ { "subnetType": "PUBLIC", "name": "Public", "cidrMask": 24 }, { "subnetType": "PRIVATE_WITH_NAT", "name": "Private", "cidrMask": 24 } ] } } }, "stg": { "vpcConfig": { "createConfig": { "vpcName": "StgVPC", "cidr": "10.101.0.0/16", "maxAzs": 2, "natCount": 2 } } }, "prd": { "vpcConfig": { "createConfig": { "vpcName": "PrdVPC", "cidr": "10.0.0.0/16", "maxAzs": 3, "natCount": 3 } } } } } Key Points: Define environment-specific parameters in JSON Use CDK's standard context section Configuration changes possible without recompilation Stack Using cdk.json Parameters // lib/stacks/cdk-json-parameters-stack.ts import * as cdk from 'aws-cdk-lib/core'; import { Construct } from 'constructs'; import { Environment } from 'lib/types'; import * as ec2 from 'aws-cdk-lib/aws-ec2'; import { pascalCase } from 'change-case-commonjs'; export interface StackProps extends cdk.StackProps { project: string; environment: Environment; isAutoDeleteObject: boolean; } export class CdkJsonParametersStack extends cdk.Stack { public readonly vpc: ec2.IVpc; constructor(scope: Construct, id: string, props: StackProps) { super(scope, id, props); // Get parameters from cdk.json const params = this.node.tryGetContext(props.environment) || {}; const vpcConfig = params['vpcConfig'] || {}; // Reference existing VPC if (vpcConfig['existingVpcId']) { this.vpc = ec2.Vpc.fromLookup(this, "VPC", { vpcId: vpcConfig['existingVpcId'], }); return; } // Check for createConfig existence if (!vpcConfig['createConfig']) { throw new Error( 'VPC createConfig is required in JSON parameters to create the VPC.' ); } const createConfig = vpcConfig['createConfig']; // Subnet configuration mapping const subnets = createConfig['subnets'] || [ { subnetType: 'PUBLIC', name: 'Public', cidrMask: 24, }, { subnetType: 'PRIVATE_WITH_NAT', name: 'Private', cidrMask: 24, } ]; // Create VPC const vpcNameSuffix = createConfig['vpcName'] ?? 'vpc'; this.vpc = new ec2.Vpc(this, "VPC", { vpcName: `${pascalCase(props.project)}/${pascalCase(props.environment)}/${pascalCase(vpcNameSuffix)}`, ipAddresses: ec2.IpAddresses.cidr( createConfig['cidr'] || '10.1.0.0/16' ), maxAzs: createConfig['maxAzs'] || 3, natGateways: createConfig['natCount'] || 1, subnetConfiguration: subnets.map((subnet: any) => { // Convert string subnetType to ec2.SubnetType if (subnet['subnetType'] === 'PUBLIC') { return { subnetType: ec2.SubnetType.PUBLIC, name: subnet['name'] || 'Public', cidrMask: subnet['cidrMask'] || 24, }; } else if (subnet['subnetType'] === 'PRIVATE_WITH_NAT') { return { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, name: subnet['name'] || 'Private', cidrMask: subnet['cidrMask'] || 24, }; } return null; }).filter((config: any) => config !== null), }); } } Key Points: Retrieve values from cdk.json using this.node.tryGetContext() Convert string-type parameters to TypeScript types Default values allow operation with minimal parameters Type checking performed at runtime Pros and Cons of cdk.json Approach Pros: No Recompilation: Deploy immediately after parameter changes CDK Standard: Standard CDK approach External Tool Integration: Easy to read with JSON parsers Low Learning Cost: Only JSON knowledge required Dynamic Values: Retrieve and calculate values at runtime Cons: No Type Safety: Cannot detect configuration errors until runtime Limited IDE Support: No autocomplete or refactoring Complex Logic: Difficult to perform calculations or conditional branching Error Handling: Runtime error handling required Which Approach Should You Choose? Approach Comparison Table Aspect TypeScript Approach cdk.json Approach Type Safety ✅ Compile-time type checking ❌ Cannot detect until runtime IDE Support ✅ Autocomplete and refactoring ⚠️ Limited Ease of Change ⚠️ Recompilation required ✅ No recompilation needed Complex Logic ✅ Easy calculation and conditional branching ❌ Difficult External Tool Integration ⚠️ Build required ✅ Easy with JSON parser Learning Cost ⚠️ TypeScript knowledge required ✅ JSON only Initial Setup ⚠️ Type definitions and file structure needed ✅ Simple Version Control Change history is clear Change history is clear Error Detection ✅ Compile-time ❌ Runtime CDK Standard ⚠️ Custom approach ✅ CDK standard Recommended Use Cases TypeScript Approach Recommended Large projects (many parameters and complex configurations) When type safety is important Team development (IDE support improves development efficiency) Complex logic (parameter calculations and conditional branching) Long-term operation (maintainability and readability focused) cdk.json Approach Recommended Small projects (simple configurations) When rapid changes are needed CDK beginners (limited TypeScript knowledge) CI/CD integration (configuration changes from external tools) Prototyping (rapid experimentation and validation) Hybrid Approach In production, combining both approaches can be effective. Basic Configuration: Managed in cdk.json (environment name, region, etc.) Complex Configuration: Managed in TypeScript (VPC configuration, security groups, etc.) Sensitive Information: Retrieved from AWS Secrets Manager Test Implementation The same test patterns can be applied to both approaches. Unit Tests // test/unit/cdk-parameters-stack.test.ts describe("CdkTSParametersStack Fine-grained Assertions", () => { let stackTemplate: Template; beforeAll(() => { const app = new cdk.App(); const stack = new CdkTSParametersStack(app, "CdkParameters", { project: "TestProject", environment: Environment.TEST, env: { account: '123456789012', region: 'ap-northeast-1' }, isAutoDeleteObject: true, terminationProtection: false, vpcConfig: { createConfig: { vpcName: "TestVPC", cidr: "10.1.0.0/16", maxAzs: 2, natCount: 1, subnets: [ { subnetType: ec2.SubnetType.PUBLIC, name: 'Public', cidrMask: 24 }, { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS, name: 'Private', cidrMask: 24 }, ], }, }, }); stackTemplate = Template.fromStack(stack); }); test("should create 1 VPC", () => { stackTemplate.resourceCountIs("AWS::EC2::VPC", 1); }); test("VPC should have correct CIDR block", () => { stackTemplate.hasResourceProperties("AWS::EC2::VPC", { CidrBlock: "10.1.0.0/16", }); }); }); Snapshot Tests // test/snapshot/snapshot.test.ts describe("Stack Snapshot Tests", () => { const app = new cdk.App({ context: testContext }); // Create all stacks first const stack = new CdkTSParametersStack(app, "CdkParameters", { project: projectName, environment: envName, env: defaultEnv, isAutoDeleteObject: true, terminationProtection: false, vpcConfig: envParams.vpcConfig, }); const jsonParameterStack = new CdkJsonParametersStack( app, "CdkJsonParameters", { project: projectName, environment: envName, env: defaultEnv, isAutoDeleteObject: true, terminationProtection: false, }); // Get templates after all stacks are created const stackTemplate = Template.fromStack(stack); const jsonParameterStackTemplate = Template.fromStack(jsonParameterStack); test("Complete CloudFormation template snapshot", () => { expect(stackTemplate.toJSON()).toMatchSnapshot(); expect(jsonParameterStackTemplate.toJSON()).toMatchSnapshot(); }); }); Key Points: Create all stacks before calling Template.fromStack() Use same test patterns for both approaches Snapshot tests detect unintended changes Deployment and Cleanup Deployment # Deploy TypeScript parameters version npm run stage:deploy:all --project=myproject --env=dev # Deploy cdk.json parameters version npm run stage:deploy:all --project=myproject --env=dev Cleanup # Delete all resources npm run stage:destroy:all --project=myproject --env=dev Summary In this exercise, we learned two major approaches for managing parameters in CDK. What We Learned TypeScript Approach: Development efficiency through type safety and IDE support cdk.json Approach: Flexibility and rapid changes Deployment Validation: Account ID checking and production environment confirmation Test Strategy: Test patterns applicable to both approaches Best Practices: Choosing based on project size Best Practices Leverage Type Definitions: Make full use of TypeScript's type system Default Values: Set default values for non-required parameters Implement Validation: Implement parameter validation before deployment Documentation: Clearly document parameter meanings and constraints Testing: Implement tests for parameter changes Reference Resources AWS CDK Documentation - Context Values AWS CDK Best Practices TypeScript Handbook Let's continue learning practical AWS CDK patterns through the 100 drill exercises! If you found this helpful, please ⭐ the repository! 📌 You can see the entire code in My GitHub Repository.

dev.to

Why Your B2B Tool Stack Probably Has Too Many Overlapping Solutions

I spent the last three weeks auditing enterprise AI tools for a mid-market SaaS company. The CEO asked me a simple question: "Why do we need four different tools that basically do the same thing?" He was right to be frustrated. Most organizations accumulate tools the way people accumulate subscriptions—one problem at a time, one quick fix at a time. A team needs better collaboration, so they grab Slack. Marketing needs project tracking, they buy Asana. HR wants automation, they subscribe to Zapier. Operations found this new AI tool that "claims" to save 10 hours per week. A year later, you've got 12 SaaS subscriptions, three of them do nearly identical things, and nobody knows which one is the "official" tool anymore. The Real Cost of Tool Sprawl I checked the numbers with that company. They were paying: $2,400/month on seven different collaboration and communication platforms Another $1,800/month on three overlapping automation tools Plus the hidden cost: developers and managers spending 6-8 hours per week just moving data between systems That's not innovation. That's debt. And it gets worse when you add AI tools into the mix. Everyone's selling "AI-powered" solutions right now. The problem is that "AI-powered" means different things depending on who's selling. Some tools genuinely save time. Others are just Excel with a neural network sticker on it. How I Evaluate Tools Now (The Framework I Actually Use) Before recommending anything, I ask three hard questions: 1. Does it replace something we're already doing, or does it add a genuinely new capability? If a tool does 80% of what you already have, it's not worth the switching cost. I've seen teams waste months migrating from one tool to another for a 15% efficiency gain. Spoiler: that gain disappears once you factor in training time and the inevitable bugs during transition. 2. Can you measure the impact in the first 30 days? If the vendor can't show you exactly where time or money gets saved (not in their marketing materials—in your workflow), be skeptical. I look for: Reduction in hours spent on specific tasks Elimination of manual data entry steps Fewer context switches between tools If you can't measure it, it's a bad bet. 3. Does it integrate with what you already have? API connectivity matters more than most people think. A brilliant tool that can't talk to your existing stack creates more work, not less. I always ask: "If this tool breaks tomorrow, can we extract our data in 2 hours?" If the answer is no, the integration is too tight. The AI Tool Wild West Here's what I've noticed: the B2B AI market is moving fast, but not always in good directions. Everyone's rushing to add large language models to their products. Some companies are doing it thoughtfully. Others are bolting ChatGPT onto an API and calling it innovation. The tools that actually deliver value are the ones solving a specific problem exceptionally well, not the ones trying to be "AI-powered" across 47 different use cases. When you're evaluating any new tool—AI or otherwise—ask the company to show you a case study from someone in your industry with a similar problem. If they can't, or if the case study looks suspiciously perfect, move on. What I'd Do Right Now If you're in the middle of a tool audit (like that company was), here's the pressure test I'd apply: Map what each tool actually does. Not what it claims to do—what your team actually uses it for. Identify overlaps. Be honest about them. There will be more than you think. Consolidate ruthlessly. Pick the best tool for each core function and commit to it for at least 6 months. Measure everything. Before and after. Hours saved, data quality, user adoption. Most organizations that do this realize they don't need more tools. They need to actually use the ones they have. The exciting part about enterprise AI right now isn't the technology—it's figuring out how to apply it thoughtfully in an already-complicated environment. The boring stuff (picking the right tools, integrating them well, measuring impact) is what actually moves the needle. If you're wrestling with a messy tool stack or trying to figure out whether that new AI solution is actually worth the investment, AIExpertReviewer breaks down these kinds of decisions with real numbers and practical frameworks.

dev.to

Automatically Refresh Forms After Status Changes in Model Driven Apps

When designing complex state models in Model Driven Apps, sometimes it can happen you need to "freeze" the possibility for the user to update a record data while some operation runs in background, and then make it editable again when the operation completes. A few practical examples: Document Generation and E-signature: A quote record triggers automatic PDF generation and sends it to DocuSign or Adobe Sign. While the document is being generated, sent, and awaiting signature, the contract terms should be locked. Once the signature is complete (or rejected), the form becomes editable again for next steps. Credit Check Integration: When a customer service rep creates a new customer account, an external credit check service is called in the background. The form needs to be frozen while waiting for the credit score, payment terms recommendations, and risk assessment to return. You don't want users changing customer details while the credit bureau is still processing the original information. Inventory Reservation in Order Processing: When an order is placed, a background process checks inventory across multiple warehouses and reserves stock. During this reservation process (which might involve calling external warehouse management systems), the order quantities should be frozen to prevent overselling or conflicts. Compliance and Regulatory Checks: In healthcare or financial services, when a case is submitted, it might trigger automated compliance screenings (sanctions lists, fraud detection, regulatory reporting). The case details must remain locked during these checks to maintain audit trail integrity. Data Enrichment from Third-party APIs: A lead record triggers enrichment processes pulling company data from LinkedIn, Clearbit, or Dun & Bradstreet. While these APIs are called and data is being validated and merged, you want to prevent users from manually entering conflicting information. Typically, you can leverage the out of the box statecode and statuscode fields of the interested table to model the frozen statuses. If we take for instance the first example above, a possible state model for the account table can be the following: Logical Status statecode statuscode Description New Active New The quote is being created and filled with data Submitted Inactive Submitted The quote has been submitted for the digital signature, the background operation is running Signed Active Signed The quote is open for post-signature steps (approvals, project generation, etc) Cancelled Inactive Cancelled The quote has been logically removed from the system Sent Inactive Sent Quote sent to the client Won Inactive Won The client accepted the quote Lost Inactive Lost The client has rejected the quote Where the meaning of the statecode=Inactive is "not updatable by the user". I often prefer to change the label of that state to "Read only", as I find this wording less misleading and more intuitive. If the trigger that changes the status to the inactive one comes from the form, we have a small UX problem: the form will be disabled (as required), the operation will run in the background (as required), but when the operation ends (after a couple of seconds, potentially), the user doesn't have any clue about the completeness. He needs to manually trigger the form refresh to see the changes to the current record. Of course you can leverage MDA Push Notification system in the async workflow to notify the user but... he still needs to reopen the record to see the changes. Fixing this UX issue is quite easy, you can just add a few lines of JavaScript in the form that: runs only when the status changes to the inactive one you want to monitor checks iteratively via WebApi for the status to change when the status changes, refreshes the form data. just like the following one: class Form { formType = { Create: 1, Update: 2, ReadOnly: 3, Disabled: 4, BulkEdit: 6 }; statuscode = { New: 1, Submitted: 2 } // Polling interval for status monitoring pollingInterval = null; onLoad(executionContext) { const formContext = executionContext.getFormContext(); const formType = formContext.ui.getFormType(); // Start auto-refresh monitoring if status is Initializing this.startStatusMonitoring(formContext); formContext.data.addOnLoad(() => { this.startStatusMonitoring(formContext); }); } /** * Starts monitoring the record status if it's in Initializing state * @param {object} formContext - The form context */ startStatusMonitoring(formContext) { const currentStatus = formContext.getAttribute("statuscode").getValue(); // Only start monitoring if status is Submitted if (currentStatus !== this.statuscode.Submitted) { return; } console.log("Starting status monitoring for Initializing record"); // Clear any existing interval if (this.pollingInterval) { clearInterval(this.pollingInterval); } // Start polling every 2 seconds this.pollingInterval = setInterval(async () => { try { await this.checkStatusChange(formContext); } catch (error) { console.error("Error checking status change:", error); // Continue polling even if there's an error } }, 2000); } /** * Checks if the record status has changed and refreshes the page if it has * @param {object} formContext - The form context */ async checkStatusChange(formContext) { const recordId = formContext.data.entity.getId().replace(/[{}]/g, ""); const entityName = formContext.data.entity.getEntityName(); try { // Retrieve the current statuscode from the server const result = await Xrm.WebApi.retrieveRecord(entityName, recordId, "?$select=statuscode"); const serverStatus = result.statuscode; const currentFormStatus = formContext.getAttribute("statuscode").getValue(); console.log(`Current form status: ${currentFormStatus}, Server status: ${serverStatus}`); // If status has changed from Submitted, refresh the page if (currentFormStatus === this.statuscode.Submitted && serverStatus !== this.statuscode.Submitted) { console.log("Status changed, refreshing page..."); // Clear the polling interval if (this.pollingInterval) { clearInterval(this.pollingInterval); this.pollingInterval = null; } // Refresh the current window formContext.data.refresh(); } } catch (error) { console.error("Failed to retrieve record status:", error); // Don't stop polling on API errors, as they might be temporary } } /** * Stops the status monitoring (useful for cleanup) */ stopStatusMonitoring() { if (this.pollingInterval) { clearInterval(this.pollingInterval); this.pollingInterval = null; console.log("Status monitoring stopped"); } } } Greg = window.Greg || {}; Greg.greg_quote = Greg.greg_quote || {}; Greg.greg_quote.Form = new Form(); Hope this helps you too!

dev.to

GLM-4.6V Now on SiliconFlow: Native Multimodal Tool Use Meets SoTA Visual Intelligence

TL;DR: ​GLM-4.6V​, Z.ai's latest multimodal large language model, is now ​available on SiliconFlow​. Featuring a 131K multimodal context window and native function calling integration, it delivers SoTA performance in ​visual understanding and reasoning ​​— seamlessly bridging the gap between "visual perception" ​and "executable ​action"​. The GLM-4.6V series provides a unified technical foundation for multimodal agents in real-world business scenarios. Try GLM-4.6V now and level up your*​ multimodal agents ​with ​SiliconFlow APIs*​. We are thrilled to announce ​GLM-4.6V​, Z.ai's latest multimodal foundation model designed for cloud and enterprise-grade scenarios, is now available on ​SiliconFlow​. It integrates native multimodal function calling capability and excels in ​long-context visual reasoning​, directly closing the loop from*​ perception to understanding to execution.* Now, through SiliconFlow's GLM-4.6V API, you can expect: Budget-friendly Pricing: GLM-4.6V $0.30/M tokens (input) and $0.90/M tokens (output) 131K Context Window: Enables processing lengthy industry reports, extensive slide decks, or long-form video content Seamless Integration: Instantly deploy via SiliconFlow's OpenAI-compatible API, or plug into your existing agentic frameworks, automation tools, or workflows. Whether you are building agents, workflows, or tools for: Rich-Text Content Creation: Convert papers, reports, and slides into polished posts for social media and knowledge bases Design-to-Code Automation: Upload screenshots/designs for pixel-level HTML/CSS/JS code generation Business Document Processing: ​Process reports to extract metrics and synthesize comparative tables Video Content Operations: ​Summarize, tag, and extract insights at scale Through SiliconFlow's production-ready API, you can leverage GLM-4.6V to power your multimodal agents in minutes — no cost concerns, no engineering overhead. Let's dive into the key capabilities with live demos from the SiliconFlow Platform. Key Features & Benchmark Performance In most LLM pipelines, tool calling is still text-only: even for image or document tasks, everything must be converted into text first, then back again. This process potentially leads to information loss and increases system complexity. GLM-4.6V changes this with native multimodal tool calling capability: Multimodal Input: Images, UI screenshots, and document pages can be passed directly as tool arguments, avoiding manual text conversion and preserving layout and visual cues. Multimodal Output: The model can directly interpret tool results such as search pages, charts, rendered web screenshots, or product images, and feed them back into its reasoning and final response. By closing the loop from ​perception → understanding → execution​, GLM-4.6V supports the following key features: Rich-Text Content Understanding and Creation: ​Accurately understands complex text, charts, tables, and formulas, then autonomously invokes visual tools to crop key visuals during generation, and audits image quality to compose publication-ready content perfect for social media & knowledge bases. Visual Web Search: ​Recognizes search intent and autonomously triggers appropriate search tools, then comprehends and aligns the mixed visual-textual results to identify relevant information, and finally performs reasoning to deliver structured, visually-rich answers. Frontend Replication & Visual Interaction: ​Achieves pixel-level ​replication by identifying layouts, components, and color schemes from screenshots to generate high-fidelity ​HTML/CSS/JS code​, then lets you refine it interactively—just circle an element and tell it what you want, like "make this button bigger and change it to green." Long-Context Understanding: ​Processes ~150 pages of documents, 200 slides, or a one-hour video in a single pass with its 131K context window, enabling tasks like analyzing financial reports or summarizing an entire football match while pinpointing specific goal events and timestamps. For example, when uploading two financial reports filled with numbers, tables and charts, GLM-4.6V shows outstanding visual understanding and reasoning performance. It really understood the tables and charts, reasoned over the numbers, and surfaced actionable insights on revenue growth, profitability, and market positioning. SiliconFlow Playground supports text & image inputs. Use API service for other input types. GLM-4.6V has also been evaluated across 20+ mainstream multimodal benchmarks including ​MMBench​, ​MathVista​, and ​OCRBench​, achieving SoTA performance among open-source models. It matches or outperforms comparable-scale models like​​ Qwen3-VL-235B​, ​Kimi-VL-A3B-Thinking-2506​, and Step3-321B in key capabilities: multimodal understanding, multimodal agentic tasks, and long-context processing. Techniques GLM-4.6V sets the technical foundation for multimodal agents in real-world business scenarios. To achieve this performance, GLM-4.6V introduces a comprehensive suite of innovations: Model architecture & long-sequence modeling: ​GLM-4.6V is continually pre-trained on long-context image–text data, with visual–language compression alignment (inspired by Glyph) to better couple visual encoding with linguistic semantics. Multimodal world knowledge: ​A billion-scale multimodal perception and world-knowledge corpus was introduced to enhance both basic visual understanding and the accuracy and completeness of cross-modal QA. Agentic data & MCP extensions: ​Through large-scale synthetic ​agentic training​, GLM-4.6V extends Model Context Protocol (MCP) with URL-based multimodal handling and end-to-end interleaved text–image output using a “Draft → Image Selection → Final Polish” workflow. RL for multimodal agents: ​Tool-calling behaviors are integrated into a unified ​RL objective​, and a visual feedback loop (building on UI2Code^N) lets the model use rendered results to self-correct its code and actions, pushing toward self-improving multimodal agents. Get Started Immediately Explore: Try GLM-4.6V in the SiliconFlow playground. Integrate: Use our OpenAI-compatible API. Explore the full API specifications in the SiliconFlow API documentation. import requests url = "https://api.siliconflow.com/v1/chat/completions" payload = { "model": "zai-org/GLM-4.6V", "messages": [ { "content": [ { "type": "image_url", "image_url": { "detail": "auto", "url": "https://tse4.mm.bing.net/th/id/OIP.mDDGH4uc_a7tmLFLJvKXrQHaEo?rs=1&pid=ImgDetMain&o=7&rm=3" } }, { "type": "text", "text": "What is in the picture?" } ], "role": "user" } ], "stream": True, "temperature": 1 } headers = { "Authorization": "Bearer <token>", "Content-Type": "application/json" } response = requests.request("POST", url, json=payload, headers=headers) print(response.text) Business or Sales Inquiries → Join our Discord community now → Follow us on X for the latest updates → Explore all available models on SiliconFlow →

dev.to

The DNA of Data: Objects & Arrays Masterclass

This article was originally published on www.codesyllabus.com Welcome back. In this article, we're going to dive deep into the absolute core of working with data in JavaScript: Objects and Arrays. If you've already worked with variables, you know they are great for storing a single piece of data. But in real-world applications—whether you're building an app with React, a backend with Node.js, or anything else—you will never deal with just a single isolated value. You will deal with collections of data. You need to group things. And that is exactly what Objects and Arrays allow us to do. We won't just look at the basic syntax; I want you to understand what happens "under the hood" and how to manipulate these structures like a pro. Arrays: Ordered Lists and Their Secrets Let's start with Arrays. Imagine you are building an app for the Madrid Metro. You have a list of lines. The order matters, right? Line 1 comes before Line 2. In JavaScript, we use square brackets [] to create an Array. It is, essentially, a list of values where each value has a numerical position or index. It is crucial to remember that indices start at 0. But Arrays are much more than simple lists; they are iterable objects that come with super powerful built-in methods. This article was originally published on www.codesyllabus.com The Spread Operator (...) Manipulating Arrays We rarely create an _Array _and leave it static. We need to add elements, remove them, or transform them. This is where methods like push or the modern map come into play. Notice how we use map. This is one of the most important functions in modern development, especially in React, to transform data without mutating the original array. Objects: Key-Value Pairs While _Arrays _are for ordered lists, _Objects _are for grouping data related to a specific entity. A user, a product, a configuration. We use curly braces {} and define key: value pairs. Here is something interesting: you can access properties using dot notation (user.name) or bracket notation (user['role']). Bracket notation is extremely useful when the key you want to access is stored inside a variable. Reference Types vs. Primitives This is a concept where many developers stumble. You might have noticed that I used const to declare the _Array _and the _Object _above. You might wonder: "Max, if it's const, why can I change things inside it?" Great question! This happens because Arrays and Objects are Reference Types. The variable does not store the value itself, but a memory address (a pointer) that points to where the data is. When we modify a property inside the object, we are not changing the memory address, only the data at that destination. That's why const allows it. However, this also means that if we copy an object simply by assigning it to another variable, we are copying the pointer, not the data! Modern Syntax: Spread and Destructuring Modern _JavaScript _(ES6+) gave us fantastic tools to work with these structures in a cleaner way. If you want to create a real copy of an object or combine two arrays, the Spread operator is your best friend. This article was originally published on www.codesyllabus.com Destructuring What if you only want to extract a specific property from an object and store it in a variable? _Destructuring _does this in a single line of code. Putting It All Together: Arrays of Objects Here is where theory meets reality. In almost all applications, you will work with Arrays of Objects. Think of a _JSON _response from an API. Mastering these structures is not just about learning syntax. It's about understanding how to model your data. Whether at Glovo handling menus or at Fever filtering events, you will always come back to these fundamentals. I hope this gave you a much clearer vision! Try the code in your console and see you in the next article in CodeSyllabus.com. This article was originally published on www.codesyllabus.com

dev.to

I Built a Privacy-First, Syntax-Aware Diff Tool because "Text Compare" Wasn't Enough

The Problem with "Generic" Diff Tools We’ve all been there. You have a local version of a file and a version from production (or a snippet from StackOverflow), and something is breaking. You paste them into a generic "Online Diff Checker." The result? A wall of black text with red and green backgrounds. While this works for plain English, it is terrible for code. Without Syntax Highlighting, skimming through 500 lines of JSON, Rust, or Python to find a missing bracket or a variable type change is mentally draining. Furthermore, most online tools require you to upload your data to their server to process the diff. If you are working on proprietary code, API keys, or client data, pasting that into a random web form is a security nightmare. Enter Diff Guru 2.0: Now with Language Awareness I built Diff Guru to solve the privacy issue by making it 100% client-side with unlimited merge capabilities. But today, I’m excited to announce a massive update: Language-Specific Syntax Highlighting. We now support over 20+ languages and formats. We don't just compare text; we render it with the same syntax coloring you are used to in VS Code. What’s New? We’ve added dedicated support and highlighting for: Modern Stack: TypeScript, JavaScript, React/JSX Backend: Python, Go, Rust, Java, C#, PHP, Ruby Systems: C, C++, Dockerfile Mobile: Swift, Kotlin Data/Config: JSON, XML, YAML, SQL, CSS, SCSS Why Syntax Highlighting Matters in a Diff Tool It reduces cognitive load. If you are comparing two JSON ** files, seeing keys in one color and string values in another makes the structure pop immediately. If you are comparing **Rust, seeing lifetimes and macros highlighted helps you distinguish logic changes from syntax noise. Try it out You can try the specific tools here based on your stack: 🐍 Python Diff Checker 🦀 Rust Diff Checker ☕ Java Diff Checker 🐳 Dockerfile Diff Roadmap I am currently working on adding even more languages. I built this tool to be free, fast, and private. I’d love to hear your feedback! If you find a language we are missing or a bug in the highlighting, let me know in the comments. Happy Coding!

dev.to

Women’s Premium Kurti: Elegance Refined for the Modern Woman

Women’s premium kurtis have become a defining element of contemporary ethnic fashion, offering a refined balance between tradition, comfort, and modern sophistication. Designed for women who value quality and timeless style, premium kurtis stand apart through their superior craftsmanship, thoughtful detailing, and graceful silhouettes. They effortlessly transition from everyday wear to elegant gatherings, making them a versatile addition to a modern wardrobe. The Enduring Appeal of Premium Kurtis The charm of a premium kurti lies in its ability to feel both classic and current. Rooted in traditional attire, it reflects cultural elegance while adapting seamlessly to modern lifestyles. Premium kurtis are designed with attention to fit and flow, creating a flattering look that suits a wide range of body types. Their appeal extends beyond trends, offering pieces that feel relevant season after season and resonate with women who appreciate understated luxury. Popular Styles and Silhouettes Women’s premium kurtis are available in a variety of styles that cater to different tastes and occasions. Straight-cut kurtis offer a sleek and polished appearance, while A-line and flared designs bring a softer, more feminine touch. Long kurtis with subtle detailing are perfect for formal settings, while shorter lengths with refined cuts suit casual elegance. Designers often incorporate minimal embroidery, fine prints, or delicate textures to enhance the kurti without overpowering its natural grace. Modern Variations and Design Details Modern premium kurtis reflect evolving fashion sensibilities while maintaining traditional roots. Contemporary variations include asymmetrical hems, layered panels, and fusion-inspired necklines that add visual interest. Subtle embellishments, refined color palettes, and clean tailoring define these designs, making them suitable for both professional and social settings. The focus remains on elegance and wearability, ensuring each piece feels sophisticated yet effortless. Fabrics That Define Premium Quality Fabric plays a crucial role in distinguishing a premium kurti from everyday wear. High-quality materials such as silk blends, fine cotton, linen, chiffon, and soft rayon are carefully selected for their texture, durability, and comfort. These fabrics drape beautifully and feel gentle against the skin, enhancing the overall wearing experience. The richness of the fabric not only elevates the look but also ensures longevity, making premium kurtis a worthwhile investment. Accessories That Complete the Look Accessorizing a women’s premium kurti allows for personalized styling while maintaining elegance. Delicate jewelry such as stud earrings, slim bangles, or minimalist necklaces complement the refined aesthetic. Footwear choices range from classic flats and khussas to elegant heels, depending on the occasion. Light dupattas, structured handbags, or subtle scarves can further enhance the ensemble without detracting from the kurti’s premium appeal. Everyday Wear with Elevated Style One of the greatest strengths of a premium kurti is its suitability for daily wear without compromising on sophistication. Whether worn to the office, a casual lunch, or an evening gathering, it offers comfort paired with a polished appearance. The breathable fabrics and thoughtful cuts ensure ease of movement, while the refined design keeps the look effortlessly stylish throughout the day. Conclusion: A Timeless Wardrobe Essential Women’s premium kurtis represent the perfect blend of tradition, quality, and modern design. They celebrate cultural roots while catering to contemporary tastes, making them an essential part of a well-curated wardrobe. With their elegant silhouettes, luxurious fabrics, and versatile styling potential, premium kurtis continue to redefine everyday fashion, offering women a graceful and confident way to express their personal style.

dev.to

Which casing do you prefer?

dev.to

[Boost]

Are We Losing Our Manners in Software Development? adiozdaniel ・ Dec 18 #discuss #performance #softwaredevelopment

dev.to

On Dec 11, 2025, the Next.js team announced two additional security issues affecting the React Server Components (RSC) protocol that were uncovered during follow-up analysis of the React2Shell patches: https://nextjs.org/blog/security-update-2025-12-11

How I Detected and Stopped a Real-World RCE Attack on My Next.js App (CVE-2025-55182 / React2Shell) Jerome Dh ・ Dec 13 #nextjs #security #react #react2shell Next.js Security Update: December 11, 2025 | Next.js Two additional vulnerabilities have been identified in React Server Components. Users should upgrade to patched versions immediately. nextjs.org

dev.to

Parrot Security OS Explained + Full Installation Guide

Introduction When individuals first venture into the realm of cybersecurity, the name that is frequently mentioned is Kali Linux. It serves as the apparent starting point — and indeed, it is a powerful tool. However, it can also feel cumbersome, particularly on older laptops. This is where Parrot Security OS quietly emerges as a more suitable option for many learners. Parrot is based on Debian, ensuring its stability. Unlike Kali, it does not significantly burden your machine. It comes equipped with hacking tools, forensic utilities, and privacy features such as Tor and AnonSurf, all pre-installed. There is no need for extensive searching or setup — it is ready to use as soon as you boot it up. Furthermore, this is not merely another article listing features. I will guide you through the process of installing Parrot step by step, as if we were sitting together. We will download it, configure it, and even execute a few commands together. By the conclusion of this guide, you will not only understand what Parrot is — you will have it operational. What is Parrot Security OS? So, what precisely is Parrot Security OS? Consider it a Linux distribution specifically tailored for one primary purpose — security and privacy. Being open source, it is available for anyone to use at no cost, and it has gained popularity among individuals learning ethical hacking, conducting investigations, or simply seeking greater control over their digital existence. Now, here’s the important part: Parrot does not aim to reinvent the wheel. It remains fundamentally Debian-based, yet it comes pre-equipped with numerous tools that would typically require hours to install. For instance, penetration testing tools are readily available. Do you want to scan a network or verify open ports? Completed. Need to analyze logs or recover a deleted file? Indeed, it includes forensic tools at your disposal. And if your primary concern is privacy, Tor and AnonSurf are immediately accessible upon logging in. In summary, it conserves your time. Rather than spending half a day installing additional packages, Parrot provides you with a fully functional workspace right out of the box. It feels lighter than Kali, yet it remains sufficiently robust for serious tasks. Frankly, it resembles a toolbox that does not burden you. Table of Contents -Introduction What is Parrot Security OS? Quick checklist before we start Backup any important data (if you’re installing on a real machine). A USB stick (8 GB or larger). A stable internet connection for downloading and updates. Know whether your PC uses UEFI or legacy BIOS (most modern PCs are UEFI). If dual-booting with Windows: disable Fast Startup in Windows and shrink Windows partition from Windows Disk Management first. 1) Download and verify the ISO Download the Parrot Security Edition ISO from the official site. Save it somewhere obvious (Downloads/ParrotOS.iso). After download, verify checksum so you don’t waste time on a corrupt ISO. On ***Linux* / macOS:** sha256sum ParrotOS.iso compare the printed hash with the one on the Parrot download page If they match, good. If not — re-download. Why this step? Corrupt ISOs cause installers to fail mid-install. Verifying saves headaches. System Requirements Let us be candid — not every laptop available is equipped to handle resource-intensive Linux distributions. The advantage of Parrot is its relatively low demand. You do not require a high-end gaming computer to experiment with it. Even a mid-range or older laptop can successfully boot it up. However, if you desire a setup that allows for a smooth experience — such as running Nmap, a web browser, and a terminal simultaneously without lagging — it is advisable to opt for slightly better specifications. An Intel i3 or i5 (or an equivalent AMD processor), a minimum of 4 GB of RAM, at least 40 GB of storage, and a standard modern display (1024×768 or higher) will provide a more pleasant experience. A small tip: if you are installing Parrot within a virtual machine like VirtualBox or VMware, allocate at least 2 CPU cores and 4 GB of RAM. Otherwise, you may find the performance to be sluggish. In summary, Parrot is adaptable. You can experiment with it on a low-end machine to familiarize yourself, but if you intend to utilize the tools effectively, it is wise to provide it with adequate resources. Press enter or click to view image in full size Step-by-Step: Install Parrot Security OS (practical guide) Quick checklist before we begin Backup any essential data (if you are installing on a physical machine). A USB drive (8 GB or larger). A reliable internet connection for downloads and updates. Determine whether your PC operates on UEFI or legacy BIOS (most contemporary PCs utilize UEFI). If you are dual-booting with Windows: disable Fast Startup in Windows and reduce the Windows partition using Windows Disk Management first. 1) Download and verify the ISO Obtain the Parrot Security Edition ISO from the official website. Save it in a location that is easy to find (Downloads/ParrotOS.iso). After downloading, verify the checksum to ensure you do not waste time with a corrupted ISO. On Linux / macOS: sha256sum ParrotOS.iso Press enter or click to view image in full size compare the printed hash with the one on the Parrot download page If they match, good. If not — re-download. Why this step? Corrupt ISOs cause installers to fail mid-install. Verifying saves headaches. Press enter or click to view image in full size 2) Create a bootable USB (safe methods) On Windows — use Rufus (easy) Download and open Rufus. Select your USB device. Under Boot selection choose the Parrot ISO. Partition scheme: GPT for UEFI systems, MBR for legacy BIOS. If you’re not sure, pick GPT for modern machines. File system: FAT32 (default). Click Start and wait. On macOS / Linux — use dd (careful) Find the USB device name (run before and after plugging the USB to compare): lsblk # Linux or diskutil list # macOS Assume USB is /dev/sdb (verify yours!). Then: sudo dd if=ParrotOS.iso of=/dev/sdX bs=4M status=progress oflag=sync Replace /dev/sdX with your USB device (on macOS use /dev/rdiskN sometimes). Double-check device name — dd will wipe the chosen device. 3) Boot from the USB Reboot your system and access the BIOS/UEFI settings (commonly by pressing F2, F12, Esc, or Del). Select USB as the primary boot device or opt for the temporary boot menu (often F12 on various laptops). Become a member In the event of a boot failure, verify the Secure Boot settings: if your computer enforces Secure Boot and the USB fails to boot, you may need to disable Secure Boot in the BIOS (some distributions are compatible with Secure Boot, but if it obstructs the boot process, it should be disabled). You will arrive at Parrot’s boot menu (with Live / Install options). Select Install Parrot (or the graphical installer). 4) Installer: language, keyboard, timezone The installer will prompt you to: Choose a language (English is acceptable). Select your keyboard layout (pick the one you utilize). Generated image Choose your timezone or region. Assign a hostname (for example, parrot-lab). Keep it straightforward. There’s nothing elaborate here — select what suits you best. 5) Create user & root passwords The installer will request: A standard user account (for your everyday use). A root password (for administrative access). Ensure both are secure and memorable. You will execute administrative commands using sudo from the standard user account (or switch to root using su, depending on the distribution’s behavior). A brief note: Avoid using simple passwords during your learning process — it fosters a poor habit. Press enter or click to view image in full size 6) Disk partitioning — choose guided or manual This is a crucial step. The installer typically presents options: A. Guided (recommended for beginners) Select Guided — utilize the entire disk (this action will erase all data on the disk). The installer will handle partition creation and formatting. Accept the terms and proceed. This is the quickest and safest choice if you do not require dual-booting or a custom layout. B. Manual (for advanced users / dual-boot) If you wish to retain Windows or create custom partitions, select Manual and adhere to the guidelines provided below. 7) Filesystems & encryption (optional) If you want disk encryption (LUKS), many installers provide an option during partitioning — choose encrypt the new installation. It will ask for a passphrase. Encryption is good for security but more advanced — keep a backup of the passphrase. 8) Install the base system The installer will copy files and install packages. This usually takes 10–30 minutes depending on medium and drive speed. Let it finish. Don’t remove the USB while copying is underway. 9) Install GRUB bootloader When asked to install GRUB, choose Yes. Install GRUB to your main drive (example /dev/sda) — not to a partition number like /dev/sda1. This will make Parrot bootable. 10) First reboot — remove USB When install finishes, reboot. Remove USB when prompted. Boot into your new Parrot system. If you see a menu with multiple OSes, pick Parrot. If it fails to boot: go back to BIOS and check boot order / secure boot settings. 11) First commands after login (do this immediately) Open a terminal and run: update package lists and upgrade installed packages sudo apt update && sudo apt full-upgrade -y remove unused packages and clean cache sudo apt autoremove -y && sudo apt autoclean reboot if kernel updated sudo reboot This brings your system fully up to date. 13) Dual-boot troubleshooting (quick tips) If Windows overwrote GRUB or you can’t boot Parrot, boot the USB in Try Live mode, open a terminal and run: find the root partition, e.g. /dev/sda2 lsblk mount it sudo mount /dev/sdaX /mnt sudo mount –bind /dev /mnt/dev sudo mount –bind /proc /mnt/proc sudo mount –bind /sys /mnt/sys chroot sudo chroot /mnt reinstall grub (example for BIOS systems) grub-install /dev/sda update-grub exit chroot and reboot exit sudo reboot (Replace /dev/sdaX with actual root partition.) For UEFI, you may need efibootmgr or reinstall GRUB-EFI packages. 14) Common mistakes & fixes (so you don’t panic) USB won’t boot → recreate the USB, try Rufus (Windows) or Etcher (multiplatform). Check BIOS USB boot order and Secure Boot. Installer fails during copy → verify ISO checksum and recreate USB. “No bootable device” after install → check BIOS boot order or reinstall GRUB to the right disk. Press enter or click to view image in full size 15) Short checklist for a smooth install (copy/paste) Backup files. Download ISO and verify checksum. Create bootable USB (Rufus / Etcher / dd). Disable Windows Fast Startup (if dual-booting). Boot from USB and choose installer. For beginners: choose Guided partition. Install GRUB to main drive. After first boot: sudo apt update && sudo apt full-upgrade -y. Install guest additions or extra tools if needed.

dev.to

Secure ML on AWS : Building Production Data Pipelines with S3 and Lambda

Reading time: ~15-20 minutes Level: Intermediate Prerequisites: Basic AWS knowledge, familiarity with S3 and Lambda Series: Part 2 of 4 - Read Part 1 Objective: This blog focuses on teaching core concepts and best practices for building data pipelines on AWS. While the code is functional, it's designed for learning and experimentation. For production deployments, additional hardening and testing would be needed. Welcome Back! In Part 1, we explored the AIDLC framework and AWS architecture for secure ML pipelines. Now, let's get hands-on with Phase 1: Data Collection & Preparation. What you'll build today: Encrypted S3 data lake with proper access controls Automated data validation pipeline with Lambda Data quality monitoring and alerting Complete audit trail with CloudTrail Infrastructure as Code with Terraform By the end: You'll have a production-ready data pipeline that ensures data quality and security from day one. The Data Pipeline Problem You can't build reliable ML models on unreliable data. Common issues include: Inconsistent data formats - Training fails due to schema changes Missing security controls - Sensitive data exposed No data validation - Bad data silently corrupts models Manual processes - Doesn't scale, error-prone No audit trail - Can't track data lineage The solution: An automated, secure, validated data pipeline. Architecture Overview Here's what we're building: AWS Services Used: S3: Encrypted data storage with versioning Lambda: Serverless data validation KMS: Encryption key management CloudTrail: Audit logging CloudWatch: Monitoring and alerting SNS: Notifications Terraform: Infrastructure as Code Step 1: Setting Up Encrypted S3 Buckets Why Three Buckets? Separation of concerns improves security and organization: Raw Data Bucket - Unvalidated data from sources Validated Data Bucket - Quality-checked, ready for training Model Artifacts Bucket - Trained models and metadata Note: The complete S3 configuration is shown below. For production, you might split this into separate files for better organization. Infrastructure Code Create terraform/s3-buckets.tf: # KMS key for encryption resource "aws_kms_key" "data_encryption" { description = "Encryption key for ML data pipeline" deletion_window_in_days = 10 enable_key_rotation = true tags = { Name = "ml-data-encryption-key" Environment = var.environment Purpose = "ML-DataPipeline" } } resource "aws_kms_alias" "data_encryption" { name = "alias/ml-data-encryption" target_key_id = aws_kms_key.data_encryption.key_id } # Raw data bucket resource "aws_s3_bucket" "raw_data" { bucket = "${var.project_name}-raw-data-${var.environment}-${data.aws_caller_identity.current.account_id}" tags = { Name = "ML Raw Data" Environment = var.environment DataStage = "Raw" } } # Enable versioning resource "aws_s3_bucket_versioning" "raw_data" { bucket = aws_s3_bucket.raw_data.id versioning_configuration { status = "Enabled" } } # Enable encryption resource "aws_s3_bucket_server_side_encryption_configuration" "raw_data" { bucket = aws_s3_bucket.raw_data.id rule { apply_server_side_encryption_by_default { sse_algorithm = "aws:kms" kms_master_key_id = aws_kms_key.data_encryption.arn } bucket_key_enabled = true } } # Block all public access resource "aws_s3_bucket_public_access_block" "raw_data" { bucket = aws_s3_bucket.raw_data.id block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true } # Enforce encryption and secure transport resource "aws_s3_bucket_policy" "raw_data" { bucket = aws_s3_bucket.raw_data.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Sid = "DenyUnencryptedObjectUploads" Effect = "Deny" Principal = "*" Action = "s3:PutObject" Resource = "${aws_s3_bucket.raw_data.arn}/*" Condition = { StringNotEquals = { "s3:x-amz-server-side-encryption" = "aws:kms" } } }, { Sid = "DenyInsecureTransport" Effect = "Deny" Principal = "*" Action = "s3:*" Resource = [ aws_s3_bucket.raw_data.arn, "${aws_s3_bucket.raw_data.arn}/*" ] Condition = { Bool = { "aws:SecureTransport" = "false" } } } ] }) } # Lifecycle policy to manage old versions resource "aws_s3_bucket_lifecycle_configuration" "raw_data" { bucket = aws_s3_bucket.raw_data.id rule { id = "delete-old-versions" status = "Enabled" noncurrent_version_expiration { noncurrent_days = 90 } } rule { id = "transition-to-glacier" status = "Enabled" transition { days = 30 storage_class = "GLACIER" } } } # Validated data bucket resource "aws_s3_bucket" "validated_data" { bucket = "${var.project_name}-validated-data-${var.environment}-${data.aws_caller_identity.current.account_id}" tags = { Name = "ML Validated Data" Environment = var.environment DataStage = "Validated" } } resource "aws_s3_bucket_versioning" "validated_data" { bucket = aws_s3_bucket.validated_data.id versioning_configuration { status = "Enabled" } } resource "aws_s3_bucket_server_side_encryption_configuration" "validated_data" { bucket = aws_s3_bucket.validated_data.id rule { apply_server_side_encryption_by_default { sse_algorithm = "aws:kms" kms_master_key_id = aws_kms_key.data_encryption.arn } bucket_key_enabled = true } } resource "aws_s3_bucket_public_access_block" "validated_data" { bucket = aws_s3_bucket.validated_data.id block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true } # Model artifacts bucket resource "aws_s3_bucket" "model_artifacts" { bucket = "${var.project_name}-model-artifacts-${var.environment}-${data.aws_caller_identity.current.account_id}" tags = { Name = "ML Model Artifacts" Environment = var.environment DataStage = "Models" } } resource "aws_s3_bucket_versioning" "model_artifacts" { bucket = aws_s3_bucket.model_artifacts.id versioning_configuration { status = "Enabled" } } resource "aws_s3_bucket_server_side_encryption_configuration" "model_artifacts" { bucket = aws_s3_bucket.model_artifacts.id rule { apply_server_side_encryption_by_default { sse_algorithm = "aws:kms" kms_master_key_id = aws_kms_key.data_encryption.arn } bucket_key_enabled = true } } resource "aws_s3_bucket_public_access_block" "model_artifacts" { bucket = aws_s3_bucket.model_artifacts.id block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true } Variables File Create terraform/variables.tf: variable "project_name" { description = "Project name prefix" type = string default = "ml-pipeline" } variable "environment" { description = "Environment (dev, staging, prod)" type = string default = "dev" } variable "aws_region" { description = "AWS region" type = string default = "ap-south-1" } variable "notification_email" { description = "Email for SNS notifications" type = string } data "aws_caller_identity" "current" {} Deploy the Infrastructure cd terraform # Initialize Terraform terraform init # Review the plan terraform plan -var="notification_email=your-email@example.com" # Apply the configuration terraform apply -var="notification_email=your-email@example.com" Step 2: Data Validation Lambda Function Why Validate Data? Prevent garbage in, garbage out: Catch schema changes early Detect data quality issues Ensure consistency Create audit trail Note: After deploying, check your email and confirm the SNS subscription to receive notifications. Validation Logic Create lambda/data-validation/handler.py: import json import os import boto3 import logging import pandas as pd from io import BytesIO import hashlib from datetime import datetime # Configure logging logger = logging.getLogger() logger.setLevel(logging.INFO) # AWS clients s3_client = boto3.client('s3') sns_client = boto3.client('sns') # Validation rules VALIDATION_RULES = { 'required_columns': ['timestamp', 'feature_1', 'feature_2', 'target'], 'numeric_columns': ['feature_1', 'feature_2', 'target'], 'max_null_percentage': 0.05, # 5% max 'min_rows': 5, # Adjusted for testing - use 100+ for production 'max_rows': 1000000, 'date_columns': ['timestamp'] } def calculate_checksum(data: bytes) -> str: """Calculate SHA256 checksum""" return hashlib.sha256(data).hexdigest() def validate_schema(df: pd.DataFrame) -> dict: """Validate DataFrame schema""" issues = [] # Check required columns missing_cols = set(VALIDATION_RULES['required_columns']) - set(df.columns) if missing_cols: issues.append(f"Missing columns: {missing_cols}") # Check numeric columns for col in VALIDATION_RULES['numeric_columns']: if col in df.columns: if not pd.api.types.is_numeric_dtype(df[col]): issues.append(f"Column '{col}' should be numeric") # Check date columns for col in VALIDATION_RULES['date_columns']: if col in df.columns: try: pd.to_datetime(df[col]) except: issues.append(f"Column '{col}' should be valid datetime") return { 'valid': len(issues) == 0, 'issues': issues } def validate_data_quality(df: pd.DataFrame) -> dict: """Validate data quality""" issues = [] # Check row count if len(df) < VALIDATION_RULES['min_rows']: issues.append( f"Insufficient rows: {len(df)} < {VALIDATION_RULES['min_rows']}" ) elif len(df) > VALIDATION_RULES['max_rows']: issues.append( f"Too many rows: {len(df)} > {VALIDATION_RULES['max_rows']}" ) # Check null values for col in df.columns: null_pct = df[col].isnull().sum() / len(df) if null_pct > VALIDATION_RULES['max_null_percentage']: issues.append( f"Column '{col}' has {null_pct:.2%} nulls " f"(max: {VALIDATION_RULES['max_null_percentage']:.2%})" ) # Check duplicates duplicate_count = df.duplicated().sum() if duplicate_count > 0: issues.append(f"Found {duplicate_count} duplicate rows") # Data distribution checks stats = {} for col in VALIDATION_RULES['numeric_columns']: if col in df.columns: stats[col] = { 'mean': float(df[col].mean()), 'std': float(df[col].std()), 'min': float(df[col].min()), 'max': float(df[col].max()), 'null_count': int(df[col].isnull().sum()) } return { 'valid': len(issues) == 0, 'issues': issues, 'stats': { 'row_count': len(df), 'column_count': len(df.columns), 'duplicate_count': duplicate_count, 'column_stats': stats } } def send_notification(topic_arn: str, subject: str, message: str): """Send SNS notification""" try: sns_client.publish( TopicArn=topic_arn, Subject=subject, Message=message ) logger.info(f"Sent notification: {subject}") except Exception as e: logger.error(f"Failed to send notification: {e}") def lambda_handler(event, context): """ Lambda handler triggered by S3 upload """ try: # Parse S3 event record = event['Records'][0] bucket = record['s3']['bucket']['name'] key = record['s3']['object']['key'] logger.info(f"Processing: s3://{bucket}/{key}") # Download file response = s3_client.get_object(Bucket=bucket, Key=key) file_content = response['Body'].read() # Calculate checksum checksum = calculate_checksum(file_content) # Load data df = pd.read_csv(BytesIO(file_content)) logger.info(f"Loaded {len(df)} rows, {len(df.columns)} columns") # Run validations schema_result = validate_schema(df) quality_result = validate_data_quality(df) # Compile results validation_result = { 'file': f"s3://{bucket}/{key}", 'timestamp': datetime.utcnow().isoformat(), 'checksum': checksum, 'schema_validation': schema_result, 'quality_validation': quality_result, 'status': ( 'PASSED' if schema_result['valid'] and quality_result['valid'] else 'FAILED' ) } logger.info(f"Validation status: {validation_result['status']}") # Save validation report report_key = key.replace('raw/', 'reports/').replace('.csv', '_report.json') s3_client.put_object( Bucket=bucket, Key=report_key, Body=json.dumps(validation_result, indent=2), ServerSideEncryption='aws:kms' ) # If passed, copy to validated bucket if validation_result['status'] == 'PASSED': validated_bucket = bucket.replace('raw-data', 'validated-data') validated_key = key.replace('raw/', 'validated/') s3_client.copy_object( CopySource={'Bucket': bucket, 'Key': key}, Bucket=validated_bucket, Key=validated_key, ServerSideEncryption='aws:kms' ) logger.info(f"Copied to: s3://{validated_bucket}/{validated_key}") # Send success notification send_notification( os.environ['SNS_TOPIC_ARN'], f' Data Validation Passed: {key}', f"File validated successfully\n\n" f"Stats:\n" f"- Rows: {quality_result['stats']['row_count']}\n" f"- Columns: {quality_result['stats']['column_count']}\n" f"- Duplicates: {quality_result['stats']['duplicate_count']}\n\n" f"Validation report: s3://{bucket}/{report_key}" ) else: # Send failure notification all_issues = schema_result['issues'] + quality_result['issues'] send_notification( os.environ['SNS_TOPIC_ARN'], f' Data Validation Failed: {key}', f"Validation issues found:\n\n" + "\n".join(f"- {issue}" for issue in all_issues) ) return { 'statusCode': 200, 'body': json.dumps(validation_result) } except Exception as e: logger.error(f"Error: {e}", exc_info=True) send_notification( os.environ.get('SNS_TOPIC_ARN', ''), f' Data Validation Error', f"Error processing {key}:\n{str(e)}" ) return { 'statusCode': 500, 'body': json.dumps({'error': str(e)}) } Lambda Dependencies Create lambda/data-validation/requirements.txt: pandas==2.1.0 boto3==1.28.85 Package Lambda cd lambda/data-validation # Create package directory mkdir package # Install dependencies pip install -r requirements.txt -t package/ # Copy handler cp handler.py package/ # Create deployment package cd package zip -r ../function.zip . cd .. Step 3: Lambda Infrastructure Create terraform/lambda.tf: # IAM role for Lambda resource "aws_iam_role" "data_validation" { name = "${var.project_name}-data-validation-lambda" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Action = "sts:AssumeRole" Effect = "Allow" Principal = { Service = "lambda.amazonaws.com" } }] }) } # IAM policy for Lambda resource "aws_iam_role_policy" "data_validation" { name = "${var.project_name}-data-validation-policy" role = aws_iam_role.data_validation.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Effect = "Allow" Action = [ "s3:GetObject", "s3:PutObject" ] Resource = [ "${aws_s3_bucket.raw_data.arn}/*", "${aws_s3_bucket.validated_data.arn}/*" ] }, { Effect = "Allow" Action = [ "kms:Decrypt", "kms:GenerateDataKey" ] Resource = aws_kms_key.data_encryption.arn }, { Effect = "Allow" Action = "sns:Publish" Resource = aws_sns_topic.validation_notifications.arn }, { Effect = "Allow" Action = [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ] Resource = "arn:aws:logs:*:*:*" } ] }) } # SNS topic for notifications resource "aws_sns_topic" "validation_notifications" { name = "${var.project_name}-validation-notifications" } resource "aws_sns_topic_subscription" "email" { topic_arn = aws_sns_topic.validation_notifications.arn protocol = "email" endpoint = var.notification_email } # Lambda function resource "aws_lambda_function" "data_validation" { filename = "${path.module}/../lambda/data-validation/function.zip" function_name = "${var.project_name}-data-validation" role = aws_iam_role.data_validation.arn handler = "handler.lambda_handler" runtime = "python3.11" timeout = 300 memory_size = 1024 source_code_hash = filebase64sha256("${path.module}/../lambda/data-validation/function.zip") environment { variables = { SNS_TOPIC_ARN = aws_sns_topic.validation_notifications.arn } } tags = { Name = "Data Validation Lambda" Environment = var.environment } } # S3 trigger resource "aws_s3_bucket_notification" "raw_data_trigger" { bucket = aws_s3_bucket.raw_data.id lambda_function { lambda_function_arn = aws_lambda_function.data_validation.arn events = ["s3:ObjectCreated:*"] filter_prefix = "raw/" filter_suffix = ".csv" } depends_on = [ aws_lambda_permission.allow_s3, aws_lambda_function.data_validation ] } resource "aws_lambda_permission" "allow_s3" { statement_id = "AllowS3Invoke" action = "lambda:InvokeFunction" function_name = aws_lambda_function.data_validation.function_name principal = "s3.amazonaws.com" source_arn = aws_s3_bucket.raw_data.arn } # CloudWatch Log Group resource "aws_cloudwatch_log_group" "data_validation" { name = "/aws/lambda/${aws_lambda_function.data_validation.function_name}" retention_in_days = 30 } Step 4: CloudTrail for Audit Logging Create terraform/cloudtrail.tf: # CloudTrail logs bucket resource "aws_s3_bucket" "cloudtrail_logs" { bucket = "${var.project_name}-cloudtrail-logs-${data.aws_caller_identity.current.account_id}" } resource "aws_s3_bucket_public_access_block" "cloudtrail_logs" { bucket = aws_s3_bucket.cloudtrail_logs.id block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true } resource "aws_s3_bucket_policy" "cloudtrail_logs" { bucket = aws_s3_bucket.cloudtrail_logs.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Sid = "AWSCloudTrailAclCheck" Effect = "Allow" Principal = { Service = "cloudtrail.amazonaws.com" } Action = "s3:GetBucketAcl" Resource = aws_s3_bucket.cloudtrail_logs.arn }, { Sid = "AWSCloudTrailWrite" Effect = "Allow" Principal = { Service = "cloudtrail.amazonaws.com" } Action = "s3:PutObject" Resource = "${aws_s3_bucket.cloudtrail_logs.arn}/*" Condition = { StringEquals = { "s3:x-amz-acl" = "bucket-owner-full-control" } } } ] }) } # CloudTrail resource "aws_cloudtrail" "data_events" { name = "${var.project_name}-data-trail" s3_bucket_name = aws_s3_bucket.cloudtrail_logs.id include_global_service_events = true is_multi_region_trail = true enable_logging = true enable_log_file_validation = true depends_on = [aws_s3_bucket_policy.cloudtrail_logs] event_selector { read_write_type = "All" include_management_events = true data_resource { type = "AWS::S3::Object" values = [ "${aws_s3_bucket.raw_data.arn}/*", "${aws_s3_bucket.validated_data.arn}/*", "${aws_s3_bucket.model_artifacts.arn}/*" ] } } tags = { Name = "ML Data Trail" Environment = var.environment } } Step 5: CloudWatch Monitoring Create terraform/monitoring.tf: # CloudWatch dashboard resource "aws_cloudwatch_dashboard" "data_pipeline" { dashboard_name = "${var.project_name}-data-pipeline" dashboard_body = jsonencode({ widgets = [ { type = "metric" x = 0 y = 0 width = 12 height = 6 properties = { metrics = [ ["AWS/Lambda", "Invocations", { stat = "Sum" label = "Lambda Invocations" dimensions = { FunctionName = aws_lambda_function.data_validation.function_name } }], [".", "Errors", { stat = "Sum" label = "Lambda Errors" dimensions = { FunctionName = aws_lambda_function.data_validation.function_name } }], [".", "Duration", { stat = "Average" label = "Avg Duration (ms)" dimensions = { FunctionName = aws_lambda_function.data_validation.function_name } }] ] period = 300 stat = "Average" region = var.aws_region title = "Lambda Metrics" view = "timeSeries" stacked = false } } ] }) } # Alarms resource "aws_cloudwatch_metric_alarm" "lambda_errors" { alarm_name = "${var.project_name}-lambda-errors" comparison_operator = "GreaterThanThreshold" evaluation_periods = "1" metric_name = "Errors" namespace = "AWS/Lambda" period = "300" statistic = "Sum" threshold = "5" alarm_description = "Lambda function errors" alarm_actions = [aws_sns_topic.validation_notifications.arn] dimensions = { FunctionName = aws_lambda_function.data_validation.function_name } } Step 6: Testing the Pipeline 1. Create Test Data Create test-data/sample.csv: timestamp,feature_1,feature_2,target 2024-01-01T00:00:00,1.5,2.3,0 2024-01-01T01:00:00,1.8,2.1,1 2024-01-01T02:00:00,1.2,2.5,0 2024-01-01T03:00:00,1.9,2.0,1 2024-01-01T04:00:00,1.4,2.4,0 2. Upload to S3 # Upload test file aws s3 cp test-data/sample.csv \ s3://ml-pipeline-raw-data-dev-YOUR_ACCOUNT_ID/raw/sample.csv \ --server-side-encryption aws:kms 3. Check Lambda Logs # View Lambda logs aws logs tail /aws/lambda/ml-pipeline-data-validation --follow 4. Verify Validated Data # Check validated bucket aws s3 ls s3://ml-pipeline-validated-data-dev-YOUR_ACCOUNT_ID/validated/ # Download validation report aws s3 cp \ s3://ml-pipeline-raw-data-dev-YOUR_ACCOUNT_ID/reports/sample_report.json \ ./ Step 7: Advanced Features Data Quality Metrics Add custom CloudWatch metrics in Lambda: import boto3 cloudwatch = boto3.client('cloudwatch') def publish_metrics(validation_result): """Publish custom metrics to CloudWatch""" metrics = [ { 'MetricName': 'DataQualityScore', 'Value': 100.0 if validation_result['status'] == 'PASSED' else 0.0, 'Unit': 'Percent' }, { 'MetricName': 'RowCount', 'Value': validation_result['quality_validation']['stats']['row_count'], 'Unit': 'Count' }, { 'MetricName': 'DuplicateRows', 'Value': validation_result['quality_validation']['stats']['duplicate_count'], 'Unit': 'Count' } ] cloudwatch.put_metric_data( Namespace='MLPipeline/DataQuality', MetricData=metrics ) Data Versioning Track data lineage: def create_data_lineage(bucket, key, checksum): """Create data lineage metadata""" metadata = { 'source_file': key, 'checksum': checksum, 'ingestion_time': datetime.utcnow().isoformat(), 'validation_version': '1.0', 'pipeline_version': os.environ.get('PIPELINE_VERSION', 'unknown') } # Store in DynamoDB or S3 s3_client.put_object( Bucket=bucket, Key=f"lineage/{checksum}.json", Body=json.dumps(metadata, indent=2) ) Important Notes Before Deploying Known Considerations 1. Lambda Package Size The pandas library is large (~100MB). For production deployments, consider using AWS Lambda Layers: # In terraform/lambda.tf, add to aws_lambda_function resource: layers = [ "arn:aws:lambda:${var.aws_region}:336392948345:layer:AWSSDKPandas-Python311:1" ] 2. Validation Rules The sample validation rules use min_rows: 5 for testing. For production: Increase min_rows to 100 or higher based on your use case Adjust max_null_percentage based on data quality requirements Customize column validation rules to match your schema 3. Terraform State Management For production, always use remote state: # Add to terraform configuration terraform { backend "s3" { bucket = "your-terraform-state-bucket" key = "ml-pipeline/terraform.tfstate" region = "ap-south-1" dynamodb_table = "terraform-state-lock" } } 4. Cost Monitoring The estimated costs (~$9.50/month) are for development with minimal usage. Always: Set up AWS Budgets and billing alerts Monitor CloudWatch costs (logs can grow quickly) Use S3 lifecycle policies to manage data retention Review CloudTrail data event costs Security Best Practices Checklist Encryption: KMS encryption for all S3 buckets Enforce encryption in bucket policies TLS in transit (enforced by bucket policy) Access Control: Least privilege IAM roles No public bucket access VPC endpoints for Lambda (optional enhancement) Audit & Compliance: CloudTrail logging all data access CloudWatch logs retention SNS notifications for failures Data Quality: Automated validation Schema enforcement Quality metrics tracking Cost Optimization Estimated Monthly Costs (Development): Service Usage Cost/Month S3 Storage (100GB) Standard ~$2.30 Lambda 10K invocations ~$0.20 CloudWatch Logs 10GB ~$5.00 CloudTrail Data events ~$2.00 Total ~$9.50 Production costs scale with data volume. Use S3 Intelligent-Tiering and lifecycle policies. Troubleshooting Guide Issue: Lambda timeout # Increase timeout in terraform/lambda.tf timeout = 900 # 15 minutes # Or reduce file size before processing Issue: Permission denied # Check IAM role has KMS permissions aws iam get-role-policy \ --role-name ml-pipeline-data-validation-lambda \ --policy-name ml-pipeline-data-validation-policy Issue: Validation always fails # Check validation rules match your data # View Lambda logs for detailed errors aws logs tail /aws/lambda/ml-pipeline-data-validation --follow What's Next? You now have: Secure, encrypted data storage Automated data validation Complete audit trail Monitoring and alerting Infrastructure as Code In Part 3, we'll build the training pipeline with SageMaker: Distributed training infrastructure Experiment tracking with MLflow Model versioning and registry Hyperparameter optimization with Spot instances In Part 4 (Series Finale), we'll complete the production setup: CI/CD pipelines for automated deployment SageMaker endpoints with auto-scaling Model monitoring and drift detection Production observability and incident response Key Takeaways Automate validation - Don't trust manual checks Encrypt everything - KMS at rest, TLS in transit Audit all access - CloudTrail is your friend Monitor data quality - Track metrics over time Use IaC - Terraform makes it reproducible Remember: Good data pipelines prevent bad models. Resources AWS Documentation: S3 Encryption Best Practices Lambda Best Practices CloudTrail Data Events Terraform AWS Provider Related Articles: Part 1: AIDLC Framework Overview Let's Connect! Questions about the data pipeline? Drop a comment below Follow me for Part 3 - SageMaker Training Like if this helped you build your pipeline Share with your team What data quality challenges are you facing? Let me know in the comments! About the Author Shoaibali MirFollow I'm a Cloud Engineer with more than 4 yrs of experience spanning across DevOps, Data Engineering, Cloud and AI/ML Engineering. Currently, am pursuing Masters Degree in AI/ML from BITS Pilani. Connect with me: Twitter/X LinkedIn Tags: #aws #machinelearning #mlops #dataengineering #terraform #lambda #s3 #cloudwatch

dev.to

The 90% Complete Trap: Why the Last 10% is Actually 90%

"How's the task progress?" "90% complete!" A week later... "What's the percentage now?" "Um... 95%." Another week later... "Still not done?" "Almost there... maybe 98%?" This is exactly the 90% syndrome. There's an old joke in software development: "The first 90% takes 90% of the time, and the remaining 10% takes another 90%." It's not a joke. It's reality. Why 90% is Actually 50% Developers make this mistake when judging progress: # Developer's mind visible_work = { "UI_implementation": 100, # ✅ Done! "basic_functionality": 100, # ✅ Done! "API_integration": 80 # Almost done } progress = sum(visible_work.values()) / 300 print(f"Progress: {progress:.0%}") # 93%! # Actual total work all_work = { "UI_implementation": 100, "basic_functionality": 100, "API_integration": 80, "edge_cases": 0, # Haven't even started "error_handling": 0, # Haven't thought about it "performance_optimization": 0, # Later... "security_verification": 0, # Was that needed? "testing": 0, # Of course later "bug_fixes": 0, # Will there be bugs? "documentation": 0 # Really necessary...? } real_progress = 280 / 700 print(f"Actual progress: {real_progress:.0%}") # 40%! Counting only visible: 90%, counting everything: 40%. Psychology of Progress Illusion 1. Ambiguous Definition of "Done" What is "done"? Developer: "Code is all written" ✅ PM: "What about tests?" Developer: "Oh... was that included?" 😅 2. The Trap of Pareto Principle The Paradox of 80/20 Rule: Implement 80% of features in 20% of time Remaining 20% of features take 80% of time The last 20% is the real deal. 3. Exponential Complexity Increase // Initially: Simple function addNumbers(a, b) { return a + b; // Done in 1 minute! } // Later: Reality function addNumbers(a, b) { // Input validation if (typeof a !== 'number' || typeof b !== 'number') { throw new Error('Invalid input'); } // Overflow check if (a > Number.MAX_SAFE_INTEGER - b) { throw new Error('Overflow'); } // Precision handling const result = parseFloat((a + b).toPrecision(15)); // Logging logger.info(`Addition performed: ${a} + ${b} = ${result}`); // Caching cache.set(`${a}+${b}`, result); return result; // Took 2 hours... } Real Case: Login Feature Actual progress report from a startup: Day 1: "50% Complete!" ✅ Login Form UI ✅ Basic Validation Day 3: "80% Complete!" ✅ Backend API ✅ Session Management Day 5: "90% Complete!" ✅ Login Success Case Day 10: "95% Complete..." ⏳ Password Recovery ⏳ Social Login ⏳ 2FA Day 15: "98% Complete..." ⏳ Browser Compatibility ⏳ Mobile Optimization ⏳ Security Vulnerability Fixes Day 20: "99% Complete..." ⏳ Load Testing ⏳ Error Messages i18n ⏳ 5 Login Failure Limit ⏳ CAPTCHA Day 25: "Really Complete!" Expected 5 days → Actual 25 days (5x) Accurate Progress Measurement Methods 1. Binary Progress (0% or 100%) The most honest method. def binary_progress(task): """Done or not done""" if task.is_completely_done(): return 100 else: return 0 # No middle ground! Advantage: Clear Disadvantage: Hard to gauge progress 2. Definition of Done Checklist Task: Login API Progress Checklist: - [ ] Feature Implementation (20%) - [ ] Unit Tests (15%) - [ ] Integration Tests (15%) - [ ] Code Review (10%) - [ ] Documentation (10%) - [ ] Security Verification (10%) - [ ] Performance Testing (10%) - [ ] Deployment Prep (10%) Completed: 3 of 8 ✓ Progress: 37.5% (Accurate!) 3. Story Point Based const storyPoints = { 'Login Form': 3, // ✅ Done 'Validation Logic': 5, // ✅ Done 'Session Management': 8, // ✅ Done 'Error Handling': 5, // ⏳ In Progress 'Security Hardening': 8, // ❌ Not Started 'Testing': 5, // ❌ Not Started }; const completed = 3 + 5 + 8; // 16 const total = 34; const progress = (completed / total) * 100; // 47% 4. Using Burndown Chart Remaining Work (hours) 100 |* 80 | * 60 | * ← Expected 40 | * * * * * ← Actual (flattened) 20 | * * * * 0 |____________________ 1 3 5 7 9 11 13 (days) The flattened section is the 90% syndrome. How to Prevent 90% Syndrome 1. Break Tasks Small ❌ "Login Feature" (40 hours) ✅ In small units: - Login Form UI (2 hours) - Email Validation (1 hour) - Password Validation (1 hour) - API Endpoint (2 hours) - JWT Token Generation (2 hours) - Session Storage (1 hour) ...Each at a completable size 2. Include Hidden Tasks in Advance def realistic_estimate(visible_work): hidden_multiplier = { "testing": 0.3, "debugging": 0.2, "refactoring": 0.1, "documentation": 0.1, "code_review": 0.1, "integration": 0.2 } total = visible_work * (1 + sum(hidden_multiplier.values())) return total # 2x 3. Use Three-Point Estimation def three_point_estimate(optimistic, realistic, pessimistic): """PERT estimation""" estimate = (optimistic + 4 * realistic + pessimistic) / 6 # Example: Login feature # Optimistic: 3 days, Realistic: 5 days, Pessimistic: 10 days # Estimate: (3 + 4*5 + 10) / 6 = 5.5 days return estimate 4. Daily Check-in ## Daily Progress Check ### What I Completed Yesterday (Specifically) - ✅ Login Form HTML/CSS - ✅ Client Validation Logic ### What I'll Do Today (Measurably) - [ ] POST /login API - [ ] JWT Token Generation ### Blockers - 🚫 Need to choose JWT library ### Actual Remaining Work (Honestly) - About 15 hours expected Response Strategies by Role Tips for PMs When you hear 90% report: "Show me the Definition of Done checklist" "Have you written tests?" "Have you handled edge cases?" "Have you updated documentation?" Most will answer "Oh..." Tips for Developers When reporting progress: "Feature is 90%, overall is 60%" "Happy Path complete, exception handling in progress" "Need 3 more days" (specific time) Tips for Management Understanding project status: Look at remaining task count rather than progress Look at burndown chart slope Check demo-able features 90% Syndrome Checklist Check if project has fallen into 90% syndrome: [ ] Progress stays in 90% range for 2+ weeks [ ] Hear "almost done" 3+ times [ ] Remaining task list keeps growing [ ] Expected completion date keeps getting pushed [ ] Team members look exhausted 3 or more? 90% syndrome. Conclusion: Honest Progress is Best "90% complete" isn't a lie. It's just a misconception. Solutions: Break tasks small Calculate hidden tasks in advance Use Binary (0/100) or checklist Daily check-in Honest communication Accepting that the last 10% is 50% of the total enables more accurate planning. Don't say "90% complete". Say "9 of 10 tasks complete". Numbers don't lie. Need accurate progress management and transparent project tracking? Check out Plexo.

dev.to

Asynchronous Warfare: How to Deliver When The Org Won't Cooperate (Part 4)

Strategies for Mocking, Decoupling, and InnerSourcing. The "Ferrari in Traffic" Problem If you followed Parts 1, 2, and 3, you now have a high-performance Tiger Team. You use Tracer Bullets to validate paths instantly. You use CoScreen/RustDesk to mob remotely with zero latency. You deploy in 15-minute increments. But then, reality hits. You need an API change from the "Core Banking Team." You open a Jira ticket. They reply: "Added to backlog. ETA: 3 weeks." Your velocity drops from 100mph to zero. This is the Asynchronous Warfare phase. Your internal speed is irrelevant if your external dependencies are blocked. You cannot "agile" your way out of a dependency on a slow team. You must Decouple. Here are the four tactical maneuvers to survive in a hostile, slow environment. Tactic 1: Mock First, Ask Later (Consumer-Driven Contracts) The Trap: You wait for the dependency to build the API so you can consume it. This is "Integration Hell" in slow motion. The Maneuver: Do not wait. Define the contract yourself. Write the OpenAPI (Swagger) spec that you wish existed. Spin up a Mock Server (using WireMock or Microcks) that returns that exact response. Build your entire feature against the Mock. Deploy it to UAT behind a Feature Flag. The Power Move: When the Core Team finally wakes up, you don't ask them "Can you design an API?" You hand them the spec and the tests: "Here is the contract we are already using. Your job is just to make the backend satisfy this test." wiremock / wiremock A tool for mocking HTTP services WireMock - flexible, open source API mocking WireMock open source is supported by WireMock Cloud. Please consider trying it out if your team needs advanced capabilities such as OpenAPI, dynamic state, data sources and more. WireMock is the popular open source tool for API mocking, with over 6 million downloads per month and powers WireMock Cloud. It can help you to create stable test and development environments isolate yourself from flaky 3rd parties and simulate APIs that don’t exist yet. Started in 2011 as a Java library by Tom Akehurst, now WireMock spans across multiple programming languages and technology stacks. It can run as a library or client wrapper in many languages, or as a standalone server. There is a big community behind the project and its ecosystem. WireMock supports several approaches for creating mock APIs - in code, via its REST API, as JSON files and… View on GitHub Benefit: You shift the power dynamic. You define the standard; they just implement it. Tactic 2: The Anti-Corruption Layer (ACL) The Trap: The Legacy Team finally delivers the API, but it's garbage. Column names are in French mixed with Hungarian notation (int_montant_ht_VAL), and the dates are strings. If you use their data structures directly, your clean code becomes "infected" by their technical debt. The Maneuver: Build a hard wall. In Domain-Driven Design (DDD), this is an Anti-Corruption Layer. Their World: Dirty Data, XML, SOAP. The Wall (ACL): A distinct Adapter/Mapper class. Your World: Clean DTOs, Immutable Records, English naming. Rule: Never let a legacy class name (e.g., T_USR_DATA_V2) leak into your domain logic. Map it at the border, even if it feels redundant. Tactic 3: InnerSource (The "Nuclear Option") The Trap: The dependency team agrees the change is easy (e.g., adding one column), but they have "no resources" to do it. The Maneuver: Stop asking for a service. Ask for access. "I understand you are busy. Give my team Write Access to your repository for 2 days. We will implement the change, write the tests, and you only have to do the Code Review." Why this works: Scenario A: They agree. You do it yourself. You are unblocked. Scenario B (Most likely): They are terrified of letting "outsiders" touch their code. Their pride gets hurt. Result: They suddenly "find time" to do it themselves by tomorrow just to keep you out. Either way, you win. Tactic 4: Weaponized Economics (For The Sponsor) The Trap: The Infrastructure Team says: "Opening this firewall port takes 10 business days. It's the SLA." The Maneuver: This is where your Executive Sponsor (from Part 3) steps in. Engineers argue about time. Managers must argue about money. Do not say: "It's too slow." Say: "The Cost of Delay for this feature is *€15,000 per week** in lost opportunity. If we wait 2 weeks for a firewall rule, are you signing off on a €30,000 loss?"* Suddenly, the firewall rule gets opened in 1 hour. Risk is the only language bureaucracy understands. Conclusion: The Completed Guide You cannot fix the entire organization. If you try to modernize everything at once, you will be crushed by the inertia. Instead, build a Tiger Team: Isolate it financially (Part 1). Equip it with tools like mr and RustDesk (Part 2 & 3). Defend it with mocks and ACLs (Part 4). Stop asking for permission to be efficient. Decouple, Mock, Deliver. This concludes "The Legacy Survival Guide". Good luck out there.

dev.to

Understanding Cross-Site Scripting (XSS): How to Detect and Prevent Attacks

Cross-Site Scripting (XSS) is a common security vulnerability found in web applications. It allows attackers to inject malicious scripts into web pages viewed by other users. These scripts can steal sensitive information, such as login credentials or personal data, and compromise the security of your website. Understanding XSS is crucial for protecting your site and its users from potential threats. This guide will help you grasp the basics of XSS, how it works, and most importantly, how you can safeguard your website against these attacks. In this blog post, we will cover: What XSS is and how it operates Different types of XSS attacks The potential impact of XSS on websites Methods to detect and prevent XSS vulnerabilities Best practices for web developers to secure their applications By the end of this post, you will have a clear understanding of XSS and the steps you need to take to enhance your website's security. What is Cross-Site Scripting (XSS)? Cross-Site Scripting (XSS) is a type of security vulnerability that allows attackers to inject malicious scripts into web pages viewed by other users. These scripts can execute in the context of the victim's browser, potentially leading to serious security issues. How XSS Attacks Work In an XSS attack, the attacker typically inserts malicious code into a web page or application. When other users visit the compromised page, the script runs in their browsers. This can enable the attacker to steal cookies, session tokens, or other sensitive data. Examples of Common XSS Attacks Session Hijacking: An attacker uses XSS to steal session cookies and impersonate a user. Phishing: Malicious scripts can create fake login forms to capture user credentials. Malware Distribution: XSS can be used to redirect users to malicious websites or download harmful software. XSS vulnerabilities can exist in any web application that processes user input without proper validation and encoding. Types of XSS Attacks There are several types of XSS attacks, each with its own methods and impacts. Understanding these types helps in identifying and mitigating XSS vulnerabilities effectively. Stored XSS Stored XSS occurs when an attacker’s script is permanently stored on a target server, such as in a database or a message forum. When other users retrieve or view this data, the script executes in their browsers. Example: An attacker posts a comment on a forum containing a malicious script. Every time someone views the comment, the script runs and can steal information or perform actions on their behalf. Reflected XSS Reflected XSS happens when an attacker’s script is included in a URL or a query parameter. The malicious code is reflected off the web server and executed immediately in the user's browser. This type of XSS often relies on tricking users into clicking malicious links. Example: An attacker sends a phishing email with a link that includes malicious script parameters. When the user clicks the link, the script executes and can steal data or perform actions. DOM-based XSS DOM-based XSS is a type of XSS where the vulnerability exists in the client-side code rather than server-side. The malicious script manipulates the Document Object Model (DOM) on the client side, executing when the web page is dynamically updated. Example: An attacker manipulates a URL parameter that causes a web page to modify its DOM in a way that executes a malicious script. How XSS Attacks Affect Websites Cross-Site Scripting (XSS) attacks can have serious consequences for websites and their users. Understanding the potential impacts helps in assessing the risk and implementing appropriate security measures. Potential Impacts on User Data When an XSS attack is successful, attackers can access sensitive user data such as login credentials, personal information, and session tokens. This data can be used for identity theft, unauthorized access, or other malicious activities. Effects on Website Functionality XSS attacks can disrupt the normal operation of a website. They can lead to unauthorized actions being performed on behalf of users, tamper with website content, or inject malware into the site, affecting all visitors. Examples of Real-World XSS Attacks MySpace Samy Worm: A famous XSS attack that spread a worm across MySpace, which modified user profiles and spread the attack further. Twitter XSS Vulnerability: Attackers used XSS to exploit Twitter's platform, leading to unauthorized access to users' accounts and sensitive data. The impact of XSS attacks can range from minor annoyances to severe breaches of privacy and security. Implementing robust security practices is essential to protect against these threats. How to Detect XSS Vulnerabilities Detecting XSS vulnerabilities is crucial for maintaining the security of your web applications. Various tools and techniques can help identify these vulnerabilities before attackers exploit them. Tools and Techniques for Detecting XSS Automated Scanners: Tools like OWASP ZAP or Burp Suite can scan your application for XSS vulnerabilities by testing various inputs and payloads. Manual Testing: Security professionals often perform manual testing by inputting various types of payloads into forms, URL parameters, and other inputs to see if they are executed by the browser. Code Review: Reviewing the source code for improper handling of user inputs and lack of proper output encoding can help identify potential XSS issues. Common Signs of XSS Vulnerabilities Unexpected Script Execution: If scripts execute in the browser after submitting user input, this may indicate a vulnerability. Unescaped Output: Check if user input is displayed on the web page without proper escaping or encoding. Errors and Warnings: Look for errors or warnings related to script injections or other unexpected behaviors. Regularly testing your web application for XSS vulnerabilities is essential to maintaining a secure environment for your users. Preventing XSS Attacks Preventing XSS attacks involves implementing various security measures to ensure that malicious scripts cannot be injected into your web application. Here are some effective strategies to protect your site: Input Validation Validating user input is a fundamental step in preventing XSS attacks. Ensure that all input data is checked against a defined set of rules or patterns before processing. Always validate and sanitize input data on both the client side and server side to minimize the risk of XSS vulnerabilities. Output Encoding Output encoding involves converting user input into a format that will not be executed as code by the browser. Use proper encoding methods to ensure that any data displayed on the web page is treated as text, not executable code. For example, use HTML entity encoding to convert special characters like `<` and `>` into their HTML equivalents to prevent script execution. Content Security Policy (CSP) Implementing a Content Security Policy (CSP) helps mitigate XSS attacks by restricting the sources from which scripts can be loaded and executed. CSP is a security layer that helps prevent unauthorized script execution. Configure CSP headers to only allow scripts from trusted sources and disallow inline scripts and eval functions. Regular Security Audits Conducting regular security audits is crucial for identifying and addressing potential vulnerabilities. Audits help ensure that your security measures are up to date and effective against new threats. Perform periodic reviews and testing of your application to identify any emerging XSS vulnerabilities. Best Practices for Web Developers Following best practices for web development helps ensure that your applications are secure against XSS and other vulnerabilities. Here are key practices to adopt: Safe Coding Practices Adopt secure coding practices to minimize the risk of XSS. This includes validating and sanitizing all user inputs, using prepared statements for database queries, and avoiding dynamic code generation. Use libraries and frameworks that have built-in protections against XSS and other common vulnerabilities. Regular Updates and Patches Keep your web application, libraries, and frameworks up to date with the latest security patches. Regular updates help protect against known vulnerabilities and ensure you have the latest security improvements. Failing to apply security patches promptly can leave your application exposed to known threats. Educating Developers about XSS Risks Educate your development team about the risks of XSS and the importance of secure coding practices. Regular training and awareness programs can help developers recognize and mitigate XSS vulnerabilities effectively. Consider incorporating security training as part of your onboarding process for new developers. Conclusion Understanding and mitigating Cross-Site Scripting (XSS) attacks is essential for maintaining the security and integrity of your web applications. By implementing the strategies and best practices outlined in this guide, you can significantly reduce the risk of XSS vulnerabilities and protect your users from potential harm. Regularly review and update your security measures to stay ahead of evolving threats and ensure a safe browsing experience for your users. FQAs What is XSS? Cross-Site Scripting (XSS) is a vulnerability that allows attackers to inject malicious scripts into web pages viewed by other users, potentially leading to data theft or other security issues. How can I prevent XSS attacks? To prevent XSS attacks, you should validate and sanitize user input, use output encoding, implement Content Security Policy (CSP), and conduct regular security audits. What are the different types of XSS attacks? The main types of XSS attacks are Stored XSS, Reflected XSS, and DOM-based XSS. Each type exploits different aspects of web applications to execute malicious scripts. How can I detect XSS vulnerabilities? You can detect XSS vulnerabilities using automated scanners, manual testing, and code reviews. Tools like OWASP ZAP and Burp Suite can help with automated scanning, while manual testing and code review help identify issues that automated tools might miss.

dev.to

A murder, a manhunt and the grandmother who wouldn't stop the search for her daughter's killer

Josephine Wentzel spent years tracking down the suspect in her daughter’s death and is now helping others grieving cold cases.

nbcnews.com

Zelenskyy says he is set to meet Trump on Sunday

Ukrainian President Volodymyr Zelenskyy says he'll probably meet with President Donald Trump on Sunday.

nbcnews.com

MinIO Alternatives (Open Source, On-Prem, Real-World Credible): SeaweedFS, Garage, RustFS, and Ceph RGW (Rook)

Introduction Choosing the right tool for your storage layer is one of the most consequential decisions in modern infrastructure. If your data footprint is large—or moving that data is expensive, slow, or operationally risky—it’s often better to pause and validate assumptions early. Rushed storage decisions can create outcomes that are not only costly and time-consuming, but sometimes genuinely hard to reverse. On the other hand, if you have multiple migration paths, you’ve assessed risks realistically, and you have rollback options if one approach fails, then a structured comparison can help you make a more informed decision. The most important principle is simple: do not decide based on feature lists alone. Decide based on your workload, your failure model, and your operational constraints—and confirm with a PoC. Criteria of This Article This article focuses on MinIO alternatives that are open source, deployable on-premise, and credible in real-world scenarios rather than just demos or marketing narratives. The options covered are SeaweedFS, Garage, RustFS, and Ceph Object Gateway (RGW) deployed on Kubernetes via Rook. Option One: SeaweedFS SeaweedFS is a distributed storage system designed to handle both small and large files, and it is often framed closer to a distributed filesystem plus object gateway approach than a pure “S3-first” product. At a high level, it combines a volume/blob layer with additional services (such as filer and gateways) to support multiple access methods, including S3 and file-oriented interfaces. In practice, SeaweedFS can be a strong fit when your workload is file-heavy or multi-access-pattern and you want architectural flexibility. It is commonly discussed as valuable for very large datasets and for workloads where colder data dominates, such as archives, backups, or large media libraries with low access frequency. The trade-off is that operational complexity can be higher than MinIO depending on how you deploy it, because the system is modular and many workflows are CLI and configuration driven rather than UI-driven. UI maturity is generally weaker than MinIO Console, so if your team prefers UI-first day-2 operations, you should plan for that gap. Authentication and IAM-like expectations are an area where the safest stance is to assume you must validate the exact auth flow you need in your environment, including the behavior of access keys and any OIDC-style integration you intend to rely on. For durability, SeaweedFS is often framed as replication-oriented for hot data, with erasure coding as a strategy you can apply for warm or cold tiers, but that typically requires deliberate lifecycle and tiering design rather than a simple toggle. Performance is also highly workload-dependent; community reports vary, so the only reliable conclusion is to benchmark your own workload with your object sizes, concurrency profile, metadata intensity, and failure scenarios. Overall, SeaweedFS can shine in file-heavy contexts and large-scale datasets, but you should expect less polished UI and more operational design effort than MinIO. Option Two: Garage Garage is a lightweight Rust-based object storage system designed with reliability and geo-distributed replication as a core focus. It frequently comes up in self-hosted discussions because it aims to be self-contained and resilient across multiple zones or sites, often on relatively modest hardware, with replication serving as the primary resilience model. The most important constraint to internalize is S3 feature completeness: Garage is unusually explicit about which parts of the S3 API it supports and which are incomplete, which is great for transparency, but it also means you should plan for the possibility that a workflow depending on advanced S3 capabilities may not work as-is. If your environment depends on specific tooling and S3 edge behaviors—such as backup tooling with strict API requirements, lifecycle policies, versioning nuances, or policy/ACL expectations—Garage should be treated as “PoC required” rather than “assume it will be compatible.” In IAM/OIDC terms, Garage is not generally positioned as an integrated IAM/OIDC console-first experience, so if you need external identity integration you will often end up building it around Garage via proxies or gateways. The same minimalism applies to UI expectations; Garage is commonly operated CLI-first, which some teams prefer, but UI-driven operations teams should plan for that operational style. Performance insights are mostly community-driven rather than benchmark-driven, and one repeatedly practical lesson is that metadata performance matters disproportionately; if metadata lands on low-IOPS storage, behavior can degrade in unpleasant ways, while placing metadata on SSD/high-IOPS storage can significantly improve stability and throughput. In summary, Garage can be a strong fit for multi-site and geo-distributed deployments when you value resilience and operational simplicity, but it demands careful validation of S3 completeness and disciplined storage choices for metadata. *Option Three: RustFS RustFS * is an open-source, S3-compatible distributed object storage system written in Rust, positioning itself around performance, simplicity, and security. It has attracted significant attention, but like any newer entrant, it should be evaluated with a production mindset that includes a real PoC, monitoring, and an explicit rollback plan. In its own documentation, RustFS presents a high-level architectural framing that distinguishes centralized-metadata and decentralized-metadata approaches and positions itself as decentralized and scalable, often alongside MinIO in that category. RustFS uses the Apache-2.0 license, which can be lower-friction for many organizations compared to copyleft licenses, particularly when internal distribution, embedding, or modifications are considerations. Operationally, one practical advantage RustFS claims over more minimal systems is a management console/dashboard, which can matter a lot for day-2 operations if your team expects visibility and routine actions through a UI. Performance is where RustFS messaging can be attractive, but it must be handled responsibly: project-published throughput figures should be treated as project-reported benchmark claims tied to specific test conditions, not as guaranteed outcomes. The only correct approach is to benchmark your own workload with realistic object distributions, concurrency, network conditions, and actual storage media (NVMe versus HDD, and the real IOPS budget). The overall picture is that RustFS may be appealing if you want an Apache-2.0, performance-oriented S3 store with a stronger UI story than minimal alternatives, but it requires more PoC discipline and version-specific validation to avoid surprises. *Option Four: Ceph RGW on Kubernetes via Rook Ceph Object Gateway (RGW) * is the object storage interface for Ceph, providing an S3-compatible REST API backed by Ceph’s storage cluster. When deployed on Kubernetes via Rook, RGW becomes especially compelling if you already operate Ceph or want a unified storage platform covering block, file, and object under one umbrella. RGW tends to be viewed as “enterprise-like” not because it is simple, but because it sits in a mature and widely deployed ecosystem with deep durability and operational concepts. That maturity comes with real trade-offs: compared to single-purpose object stores, Ceph is heavier operationally, has more moving parts, and benefits strongly from solid monitoring, capacity planning, and failure drills. Rook helps by providing Kubernetes-native deployment patterns and CRDs for Ceph components, including object stores and related workflows. If your team can afford the operational footprint, RGW is often the choice that maximizes ecosystem depth and long-term flexibility, especially when object storage is not an isolated component but part of a broader storage strategy. **What to Validate in a Real PoC A credible decision requires a measurable PoC. Define your workload concretely by testing realistic object sizes and access patterns, not just a single synthetic benchmark. At minimum, validate small-object behavior, medium objects, and large objects, because metadata and multipart behavior can dominate real performance. Test read-heavy, write-heavy, and mixed workloads, and include metadata-heavy operations such as listing and delete patterns if your applications do them. Run concurrency levels that reflect production clients, and treat latency sensitivity as a first-class requirement if your applications have tight timeouts or synchronous request patterns. Compatibility must be tested against your actual toolchain and “non-negotiable” S3 expectations, including backup tools, lifecycle rules, versioning, replication requirements, encryption expectations, and multipart uploads. Just as importantly, run failure and recovery drills: take nodes down, simulate zone loss, introduce network instability, and observe behavior while the system is rebalancing or recovering, because many systems look fine until recovery pressure reveals weak points. Finally, validate day-2 operations explicitly, including provisioning, secret and key rotation, observability (metrics/logs/alerts), upgrade procedures, and time-to-recover targets under realistic failure events.

dev.to

Mountain community in California struck by mudslides

Mountain community in California struck by mudslides

nbcnews.com

What to Know About World Models, the Tech Some Say Is the Future of AI - Business Insider

Full text is unavailable in the news API lite version

businessinsider.com

What Is LAMP Server? Components, Working & Installation Guide

LAMP represents a common open-source software stack, including Linux, Apache, MySQL (or MariaDB), and PHP (or Perl/Python). It's frequently used to run dynamic websites and web apps on Linux servers. Components of LAMP: Linux: The underlying operating system. Apache: A widely used web server that processes HTTP requests and delivers web content. MySQL/MariaDB: A relational database management system (RDBMS) for storing and managing data. PHP/Python/Perl: A programming language for server-side scripting. How LAMP Works: Linux forms the stable base for all components. Apache receives incoming HTTP requests and serves web pages. PHP (or another scripting language) processes dynamic content. MySQL/MariaDB stores and manages website data. Installing LAMP on Linux: On Ubuntu/Debian: sudo apt install apache2 mysql-server php php-mysql Enable and start services: sudo systemctl enable apache2 sudo systemctl start apache2 sudo systemctl enable mysql sudo systemctl start mysql On CentOS/Fedora: sudo apt install httpd mysql-server php php-mysql Enable and start services: sudo systemctl enable httpd sudo systemctl start httpd sudo systemctl enable mysql sudo systemctl start mysql Verify Installation: Apache: Open a web browser and go to http://your-server-ip. You should see the Apache default page. PHP: Create a test file /var/www/html/info.php with the following content: Then access http://your-server-ip/info.php in your browser. That's it for setting up a LAMP server! Contact us if you need more information.

dev.to