# Ayobami Adewole — Full Blog Content > Complete content of every post on ayobamiadewole.com, converted to Markdown. See /llms.txt for a shorter curated index. Generated at build time from 55 posts. --- ## What I've Been Building: Alehra URL: https://www.ayobamiadewole.com/Blog/what-ive-been-building-alehra/ Category: Projects Published: 2025-12-23 Summary: Alehra is a practice operating system for behavioral health providers, therapists, counselors and psychologists, built by the author alongside their day job. It began as a white-labeled deployment and now serves more than 250 providers and 3,000 patients. If you've followed this blog for a while, you've mostly seen posts about testing methodologies, CI/CD pipelines, and the occasional dive into machine learning in .NET. This post is a bit different. I want to introduce something I've been building for the past couple of years alongside my day job: Alehra, a practice operating system for health providers, therapists, counselors, and psychologists. ## Why not just use an existing EHR? Most EHR software on the market was built for general medical practices and later adapted for behavioral health. That adaptation shows up in small but constant friction: note templates that don't match how therapy sessions actually unfold, billing workflows built around procedure codes rather than session-based care, and telehealth bolted on as an afterthought rather than built in from the start. Alehra is built specifically for behavioral health from the ground up, and that shows up in a few concrete ways. Documentation is ambient rather than manual: the system listens during a session and drafts a clinical note using templates a practice can customize to match their own language and structure, instead of forcing clinicians into a rigid form. An AI agent reviews each note for clinical concerns and helps track progress against a client's treatment plan over time, something general-purpose EHRs typically don't attempt at all. Telehealth runs natively rather than through a bolted-on third-party link, and the same ambient documentation works whether a session happens in person or on screen. Billing is unified across payment providers, with funds going directly to the practitioner rather than sitting with an intermediary. The result is a single system instead of the four or five disconnected tools a lot of practices are stitching together today. ## The engineering behind it For readers who come here for the technical content, here's a bit of what's under the hood. The frontend is React, and the backend is a mix of microservices in Node.js/TypeScript and Python with FastAPI. The ambient documentation feature runs on a mix of large language models. Everything runs on GCP. Cursor with assistance through Claude Code & Codex have been my primary tools for a lot of the day-to-day implementation work, which is its own interesting story about how AI-assisted coding changes what a two-person technical team can ship. ## Where things stand Alehra started as a white-labeled deployment that grew to serve more than 250 providers and 3,000 patients. We're working through APA certification in the US and Alberta, building partnerships with the Canadian Psychological Association and the Psychologists' Association of Alberta, and starting to plan an expansion into behavioral health markets across Africa. I'll probably write more here about the technical decisions behind Alehra as we go, from real-time telehealth to how we've approached hallucination hardening in the clinical documentation pipeline. For now, if you want to see what we've built, you can find it at [alehra.com](https://alehra.com). ### Frequently asked questions **What is Alehra?** A practice operating system for health providers, therapists, counselors and psychologists that the author has been building for the past couple of years alongside a day job. You can find it at alehra.com. **How is Alehra different from a general-purpose EHR?** It is built for behavioral health from the ground up. Documentation is ambient: the system listens during a session and drafts a clinical note from templates a practice can customize. An AI agent reviews each note for clinical concerns and tracks progress against the treatment plan. Telehealth is native, and billing is unified across payment providers with funds going directly to the practitioner. **What technology is Alehra built with?** A React frontend and a backend of Node.js/TypeScript and Python (FastAPI) microservices, with ambient documentation running on a mix of large language models. Everything runs on GCP. **How big is Alehra and what is next for it?** It started as a white-labeled deployment and grew to serve more than 250 providers and 3,000 patients. The team is working through APA certification in the US and Alberta, building partnerships with the Canadian Psychological Association and the Psychologists' Association of Alberta, and planning expansion into behavioral health markets across Africa. --- ## Swarm Testing URL: https://www.ayobamiadewole.com/Blog/Swarm-Testing/ Category: Management Published: 2022-07-11 Summary: Swarm testing applies swarming (mobbing) to software testing: a team meets for a 30 to 45 minute huddle to test a feature before release, using prepared test plans and Jira tickets for bugs. It complements existing QA rather than replacing it. #### Introduction Swarming also known as mobbing or huddling is now increasingly being used in software engineering. This is a process where a team or group of people agree to collectively tackle a task, resolve an issue or incident. This approach was traditionally used when a team must complete a task given a limited time. It is known to facilitate knowledge sharing and collaboration within a team. When swarming process is applied to software testing, this is referred to as Swarm Testing. During swarm testing, a team organizes a huddle to test a feature of a software product before the eventual release. This process is not to replace but complete the existing quality assurance processes for software engineering cycle that a team might already have in place. This type of testing is effective at reducing cycle time, improving coverage and fault detection. #### Swarm Testing Session The engineer or the lead of a project is expected to consult with the team to determine if the features of the project to be tested qualifies for a swarm testing session. The project lead will identify the participants of the session and send out the meeting invites. The recommended approach is to have a mix of members that worked on the product features and those that did not. Test plans and scenarios are to be prepared ahead and allocated to the participants. The appropriate duration of the session should be determined based on the number of test plans to be covered. It is recommended to keep the session between 30 to 45 minutes for optimal focus and participation. Before each swarm testing session, the project lead will create the test plans and scenarios in a document or repository that is accessible to all the participants. The test plans should not contain tests already covered in the unit or end to end tests for the feature. A document or wiki page should be created using the template available later in this post. The swarm testing session document or wiki page should be filled by the project lead and shared with session attendees before the session. During the session, the project lead will introduce the new feature by presenting the end to end flow of the most common use cases. The role of each participant in the session is to have their environment and test accounts set up if necessary. For example, some test plans require certain access or privileges, these priviledges should be requested before the session. Any participant can create the Jira ticket for any issue or bug detected during the session. The jira ticket created should include a detailed description of what the issue is, steps to reproduce and the screenshot of the issue if applicable. #### Swarm Testing Template This template is a guideline that could be used in preparation for a swarm testing session. The template provides details about the session and guideline for participants on what to focus on during the session and where to log the outcome of the tests. **Project Details** **Project:** < Link to project page > **Feature Flag:** < Feature Flag > **Date & Time:** < Date & Time > **Product & Technical Documents** < Link to Docs > **Design Assets:** < Link to Figma > **Jira Epic** < Link to Jira Epic > **Attendees** **Project Lead(s):** < Link to Profile > **Team members that worked on the feature:** < Link to Profile > **Other team members:** < Link to Profile > **EM, Design & PM:** < Link to Profile > **Environment Setup** These are the requirements to be met by each participant for the session. ##### Tools & Access: * dev/staging/prod access * Access to Jira backlog ##### Test Accounts: * Brand-new account, no data * Account with some test data * Account with lots of test data ##### Browsers: * Google Chrome * Safari * Firefox * iOS/Android browser **Test Plans** **For PM, Design & EM** Focus on design patterns and user flow < Link to test plan > < Link to test plan > **Other team members:** < Link to test plan > < Link to test plan > **Team members that worked on the feature:** < Link to test plan > < Link to test plan > **Filing bug or issues** Participants will update this section with links to jira bug tickets, for PM to prioritize. **Jira Bug Ticket Link** **Check if Blocking Release** < Link to Jira ticket created > < Link to Jira ticket created > table, th, td { border: 1px solid black; border-collapse: collapse; padding: 5px; margin: 5px; } ### Frequently asked questions **What is swarm testing?** When a team organizes a huddle to collectively test a feature of a software product before release, applying swarming (also called mobbing or huddling) to testing. It is meant to complete, not replace, the QA processes a team already has. **How long should a session be and who should attend?** The recommended length is 30 to 45 minutes for optimal focus and participation. Invite a mix of team members who worked on the feature and those who did not, plus engineering management, design and product. **How are test plans and bugs handled?** The project lead prepares test plans and scenarios in advance in a shared document or repository, excluding tests already covered by unit or end-to-end tests. Any participant can create a Jira ticket for a problem, with a description, steps to reproduce and a screenshot if applicable. --- ## Agile Story Estimation URL: https://www.ayobamiadewole.com/Blog/Story-Estimation/ Category: Management Published: 2022-05-28 Summary: Story points measure a story's relative effort and complexity, commonly on a Fibonacci scale (1, 2, 3, 5, 8, 13...). Teams size stories in backlog refinement with planning poker, discuss differing estimates, agree on a number, and split stories above the agreed limit. **Introduction** Agile story estimations can be done through the use of time or story points. The story points approach is popular and widely used by team because the story points are assigned to a story in the backlog as a measure of relative effort and complexity of that story. This blog post intent is not to teach the team to achieve perfect or accurate estimation, but to assist the team in achieving reliable velocity. With reliable velocity, the team can easily predict how long it would take to deliver a unit of work in the backlog. A unit of work could be a milestone or an epic. **Velocity** A team’s velocity is a measurement of the number of estimation units that the team completes sprint after sprint. The velocity is based on the number of stories that have successfully been completed in the past. This means when starting a new sprint, the team is able to use the velocity metrics as an indication to determine the number of stories in the backlog that can be realistically completed in the sprint. Additionally, when the team has reliable velocity, the team can confidently predict how long it will take to deliver a milestone, while putting other factors into consideration ( PTO, On-point/On-Call e.t.c). **Story point size** The size of stories is estimated using the Fibonacci Agile Estimation approach, which uses the Fibonacci sequence as a scale for estimating story points. The Fibonacci sequence is as follows: 1, 2, 3, 5, 8, 13, 21, 34, 55, 89… and so on, where each number is the sum of the preceding numbers. In essence, a task estimated as 2 story points is expected to be twice as complex and requires double efforts to complete as a task estimated as 1 story point. Also, 3 story points should equal triple the effort and complexity compared to 1 story point. A team can adopt a story point sizing approach where 1 is the minimum point and 5 is the maximum point that can be assigned to any story. If there is a story bigger than the agreed limit (8, 13, or more) then it should be split into smaller stories or tasks. Smaller stories facilitate precise estimation. For example, a story to change the copy of a button in a react component might be assigned a 1 story point. While a story to update routing for the some backend endpoints might be assigned a 5 story point, because of the effort and the complexity required to complete the story. **Assigning story point** Ideally a team will size the stories in the backlog during the backlog grooming or refinement session. During this session, the backlog items are reviewed, discussed, sized and prioritized. When attempting to estimate a story, the team will discuss some questions related to the story, such as: 1. How much work is involved? 2. How complex is the work? 3. Will the team be able to complete the story given the current technical abilities? 4. What are the potential risks? 5. Are there requirements that need further clarification or investigation? 6. Will the team require collaboration or help from other teams? 7. Have we considered everything in the definition of done? A planning poker agile estimation technique can be used to size each story. Each team member will provide a Fibonacci number ( 1, 2, 3, 5, e.tc.) that depicts the size of the story being reviewed, based on the complexity and required effort to implement. The process will not involve the use of actual cards, but team members can send in the story points to the message channel used for the grooming session, in a coordinated voting manner. **How is it done?** For example, let's assume in a backlog grooming session, the team is trying to size a story – ‘Migrate logs to Splunk’. The team will have to discuss the story based on the questions discussed above. After the efforts, complexity and uncertainty have been determined, the team can proceed to attempt estimation, using the planning poker approach. If there are three members in the grooming session, the team can have the same story estimated differently as below: 1. Estimation by first member = 1 2. Estimation by second member = 2 3. Estimation by third member = 5 The estimation from the first and second team members are close, but the third team member gave the story a 5 point. The third team member would explain why they think the story is bigger. The team will discuss, review and eventually agree on the appropriate point to assign to the story before moving on to estimate the next story in the backlog. This process is repeated until all the stories in the backlog has been groomed. ### Frequently asked questions **What is a story point?** A measure of the relative effort and complexity of a story in the backlog, used as an alternative to estimating in units of time. **Why is the Fibonacci sequence used for estimation?** The Fibonacci Agile Estimation approach uses the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89...) as the scale for story points. **What should a team do with a story that is too big?** If a story exceeds the agreed limit (such as 8 or 13), it should be split into smaller stories or tasks, since smaller stories allow more precise estimation. **What is velocity?** The number of estimation units a team completes sprint after sprint, based on the stories it has completed in the past. Teams use it to plan how much to take on in a new sprint. --- ## Django deployment to Google App Engine with Github Action URL: https://www.ayobamiadewole.com/Blog/Github-Actions/ Category: Python Published: 2021-02-01 Summary: A GitHub Action can deploy a Django app to Google App Engine on every push to main. The repo needs main.py, requirements.txt and app.yaml at its root, plus a workflow in .github/workflows that checks out the code and deploys it. **Introduction** Github action was released by Github as a feature to enable automation and customization of the development to deployment workflows for software development teams. Github actions ease the flow of execution, testing and possibly deployment of repository residing on Github. Also, Github actions provide the flexibility of creating custom flows and steps that are available in standards Continuous Integration (CI) and Continuous Deployment (CD) solutions. Additionally, using Github actions,process or flow of reviewing codes, triaging issues and managing repository branches and updating dependent libraries in projects become very easy and seamless. The full release note and information is available here [https://github.com/features/action](https://github.com/features/action) **Deploying Django To Google App Engine.** Let's explore how Github actions can be used to automate deployment of a Django application hosted on Github to Google App Engine. There are different approaches to setting up Github actions for your repository and a repository can have multiple actions setup and hooked to different events. A Github action could be set up to listen to pull request merge events to a main branch, and then proceed to build, test and deploy to any of the cloud providers of choice. In order to ensure a Django app can be deployed from Github to Google app engine, there are certain considerations and setup procedures. The root folder of your Django application should contain the following files main.py, requirements.txt and app.yaml See a sample App.yaml file below # App.yaml file # https://cloud.google.com/appengine/docs/standard/python3/config/appref runtime: python39 instance\_class: F2 env\_variables: DJANGO\_SETTINGS\_MODULE: "settings" DB\_NAME: "db\_name" DB\_USERNAME: "db\_user" DB\_PASSWORD: "db\_password" DB\_HOST: "/cloudsql/double-media-xxxx:europe-west1:db\_name" DB\_PORT: 3306 DB\_TIMEOUT: 5000 handlers: # This configures Google App Engine to serve the files in the app's static # directory. - url: /static static\_dir: static/ # This handler routes all requests not caught above to your main app. It is # required when static routes are defined, but can be omitted (along with # the entire handlers section) when there are no static files defined. - url: /.\* secure: always redirect\_http\_response\_code: 301 script: auto Then proceed to create .github/workflows folder in your repository root folder and create a github action to subscribe to the push event on the main branch, see the sample action file below. # .github/workflows/deploy # https://cloud.google.com/python/django/appengine # https://github.com/actions-hub/gcloud: name: deploy-app-to-gcp on: push: branches: \[ main\] paths: - 'app/\*\*' jobs: deploy: name: Deploy runs-on: ubuntu-latest steps: - uses: actions/checkout@v1 - uses: actions-hub/gcloud@master env: PROJECT\_ID: ${{secrets.GCLOUD\_PROJECT\_PROD\_ID}} APPLICATION\_CREDENTIALS: ${{secrets.GCLOUD\_GITHUB\_CREDENTIALS}} with: args: app deploy ./app.yaml The github action posted above has depends on two other actions, the first to checkout the latest code and then deploy the Django app to Google app engine, using the credentials and project id that can be set in Github secrets. ### Frequently asked questions **What files does a Django app need to deploy to Google App Engine?** The root folder of the Django application should contain main.py, requirements.txt and app.yaml. **How do I trigger the deployment from GitHub?** Create a .github/workflows folder in the repository root and add a workflow that subscribes to the push event on the main branch. It uses two other actions: one to check out the latest code and one to deploy the app to App Engine. **Where do the Google Cloud credentials go?** The credentials and project id are stored as GitHub secrets, which the deploy action uses. --- ## Named entity extraction from text in Python URL: https://www.ayobamiadewole.com/Blog/named-entity-extraction-from-text-in-python/ Category: Python Published: 2020-01-12 Summary: Named entity recognition finds and classifies entities such as people, places, organizations, dates and money in text. The post demonstrates it with spaCy, whose models were trained on the OntoNotes 5 corpus, and compares spaCy with NLTK. **Introduction** The ability to understand and process natural language is regarded as one of the superiority of humans to computer. Humans if able to read and or possibly write, can readily comprehend written texts and make meanings out of the syllables, grammar, vocabulary and punctuation. Although, as simple and effortless as it might seem for humans, making meaning out of a simple text for computers requires a bit of rules, logic driven approach or machine training and adaptation. Natural Language Processing (NLP) in Python has become commonplace, thanks to the availability of mature, open source and industrial grade readily available libraries with little learning curve. Out of the open source NLP libraries available for Python, Natural Language Toolkit (NLTK) and spaCy stand out with huge community engagement and resources. **NLTK vs spaCy** NLTK is a suite of open source Natural Language Processing library containing Python modules with extensive data sets and examples, with great support for research and development. Also, NLTK provides easy-to-use interfaces to numerous corpora and lexical resources such as WordNet. It includes a suite of text processing libraries for performing classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers on natural languages. The github code is at [https://github.com/nltk/nltk](https://github.com/nltk/nltk). spaCy on the other is more suited for large scale Natural Language Processing tasks and is commonly used in industry and enterprise applications with NLP requirements, with support for over 50 languages. Spacy can be used together with any of Python’s AI libraries, it works seamlessly with TensorFlow, PyTorch, scikit-learn and Gensim. It is fairly easier to build linguistically advanced statistical models for a variety of NLP problems using spaCy compared to NLTK. More info on spacCy can be found at [https://spacy.io/](https://spacy.io/). **Named Entity Recognition** Named entity recognition (NER) is a subset or subtask of information extraction. It involves identifying and classifying named entities in text into sets of pre-defined categories. These categories include names of persons, locations, expressions of times, organizations, quantities, monetary values and so on. NER has real word usages in various Natural Language Processing problems. Some sample usages might include * Building a minimalistic search engine, you might want to identify locations, names or even products in search texts. * Tweet mining, to determine if it contains locations or persons of interests. **Named Entity Recognition using spaCy** spaCy named entity recognition models were trained using OntoNotes 5 corpus and these models support the following entity types: Type Description PERSON People, including fictional. NORP Nationalities or religious or political groups. FAC Buildings, airports, highways, bridges, etc. ORG Companies, agencies, institutions, etc. GPE Countries, cities, states. LOC Non-GPE locations, mountain ranges, bodies of water. PRODUCT Objects, vehicles, foods, etc. (Not services.) EVENT Named hurricanes, battles, wars, sports events, etc. WORK\_OF\_ART Titles of books, songs, etc. LAW Named documents made into laws. LANGUAGE Any named language. DATE Absolute or relative dates or periods. TIME Times smaller than a day. PERCENT Percentage, including ”%“. MONEY Monetary values, including unit. QUANTITY Measurements, as of weight or distance. ORDINAL “first”, “second”, etc. CARDINAL Numerals that do not fall under another type. Install spaCy library with pip and download the english model using the commands below from terminal. pip install spacy python -m spacy download en We will be using text “John bought a Toyota camry 2019 model in Toronto in January 2020 at a cost of $38000” and pass this to spacy to recognize the entities in the text. Recognizing entities in a text using spaCy involves three steps. **First step**: Loading the spaCy library spacy\_nlp = spacy.load('en\_core\_web\_sm') If you are using spaCy medium or large pre-trained model, use 'en\_core\_web\_md' or 'en\_core\_web\_lg' insead of 'en\_core\_web\_sm'. **Second step**: Constructing a spaCy document from the text text = "John bought a Toyota camry 2019 model in Toronto in January 2020 at a cost of $38000" doc = spacy\_nlp(text.strip()) **Third step**: The doc object constructed contains loaded entities which can be accessed to extract the entities recognized text = "John bought a Toyota camry 2019 model in Toronto in January 2020 at a cost of $38000" doc = spacy\_nlp(text.strip()) The snippet below shows how the entities are extracted using the spaCy supported entity types. # create sets to hold words named\_entities = set() money\_entities = set() organization\_entities = set() location\_entities = set() time\_indicator\_entities = set() for i in doc.ents: entry = str(i.lemma\_).lower() text = text.replace(str(i).lower(), "") # Time indicator entities detection if i.label\_ in \["TIM", "DATE"\]: time\_indicator\_entities.add(entry) # money value entities detection elif i.label\_ in \["MONEY"\]: money\_entities.add(entry) # organization entities detection elif i.label\_ in \["ORG"\]: organization\_entities.add(entry) # Geographical and Geographical entities detection elif i.label\_ in \["GPE", "GEO"\]: location\_entities.add(entry) # extract artifacts, events and natural phenomenon from text elif i.label\_ in \["ART", "EVE", "NAT", "PERSON"\]: named\_entities.add(entry.title()) The recognized entities can be printed to the console. print(f"named entities - {named\_entities}") print(f"money entities - {money\_entities}") print(f"location entities - {location\_entities}") print(f"time indicator entities - {time\_indicator\_entities}") print(f"organization entities - {organization\_entities}") ------------------------------------------------------------- named entities - {'John'} money entities - {'38000'} location entities - {'toronto'} time indicator entities - {'2019', 'january 2020'} organization entities - {'toyota'} In the output John was extracted as the named entity, 38000 as moeny entity, Toronto as location entity, Toyota as organization entity, lastly 2019 and Janauary 2020 as time indicator entities. Complete source code listing is below. import spacy class NamedEntityExtractor: """ Performs named entity recognition from texts """ def extract(self, text: str) : """ Performs named entity recognition from text :param text: Text to extract """ # load spacy nlp library spacy\_nlp = spacy.load('en\_core\_web\_sm') # parse text into spacy document doc = spacy\_nlp(text.strip()) # create sets to hold words named\_entities = set() money\_entities = set() organization\_entities = set() location\_entities = set() time\_indicator\_entities = set() for i in doc.ents: entry = str(i.lemma\_).lower() text = text.replace(str(i).lower(), "") # Time indicator entities detection if i.label\_ in \["TIM", "DATE"\]: time\_indicator\_entities.add(entry) # money value entities detection elif i.label\_ in \["MONEY"\]: money\_entities.add(entry) # organization entities detection elif i.label\_ in \["ORG"\]: organization\_entities.add(entry) # Geographical and Geographical entities detection elif i.label\_ in \["GPE", "GEO"\]: location\_entities.add(entry) # extract artifacts, events and natural phenomenon from text elif i.label\_ in \["ART", "EVE", "NAT", "PERSON"\]: named\_entities.add(entry.title()) print(f"named entities - {named\_entities}") print(f"money entities - {money\_entities}") print(f"location entities - {location\_entities}") print(f"time indicator entities - {time\_indicator\_entities}") print(f"organization entities - {organization\_entities}") if \_\_name\_\_ == '\_\_main\_\_': named\_entity\_extractor = NamedEntityExtractor() text = "John bought a Toyota camry 2019 model in Toronto in January 2020 at a cost of $38000" named\_entity\_extractor.extract(text) Named entity extraction has numerous real world usages, with samples mentioned earlier. spaCy can easily be boostrapped to add this fucntionality to your enterprise application. ### Frequently asked questions **What is named entity recognition?** A subtask of information extraction that identifies named entities in text and classifies them into predefined categories such as persons, locations, times, organizations, quantities and monetary values. **How do NLTK and spaCy differ?** NLTK is a suite of Python modules with extensive datasets and examples, with great support for research and development. spaCy is better suited to large-scale, industry and enterprise NLP, supports over 50 languages, and works with TensorFlow, PyTorch and scikit-learn. **How do I extract entities with spaCy?** Three steps: load the spaCy library and a model, construct a spaCy document from the text, then read the recognized entities from the doc object. In the post's example it picked out a person, a money amount, a location, an organization and dates. --- ## Log Analytics in Azure URL: https://www.ayobamiadewole.com/Blog/Log-Analytics-in-Azure/ Category: Others Published: 2019-03-19 Summary: An excerpt from Ritesh Modi's Azure for Architects. Log Analytics (OMS) collects and searches telemetry from VMs and containers through installed agents, with no application code changes, and supports solutions and alert rules with email, webhook, runbook or ITSM actions. **Introduction** _Learn about Log Analytics in this article by Ritesh Modi, an ex-Microsoft Senior Technology Evangelist. He is a Microsoft Regional Director and the Regional Lead for Microsoft certified trainers._ Log Analytics service helps in providing information about environments using its rich log storage and analytic features. Application Insights is used for monitoring applications. However, it is equally important to monitor the environment on which an application is hosted and running. This involves the infrastructure, such as virtual machines, Docker containers, and related components. **Provisioning** Log Analytics, also known as the **Operational Management Suite (OMS)**, must be provisioned on Azure before it can be used to monitor virtual machines and containers. Again, similar to Application Insights, Operational Insights can be provisioned through the Azure Portal, PowerShell, REST APIs, and resource group manager templates. An Operational Insights workspace is a security boundary that can be accessed by certain users. Multiple workspaces should be created to isolate users and their corresponding access to environment telemetry data. The JSON script used to provision an Operational Insights workspace is shown here: { "apiVersion": "2015-11-01-preview", "type": "Microsoft.OperationalInsights/workspaces", "name": "\[parameters('workspaceName')\]", "location": "\[parameters('deployLocation')\]", "properties": { "sku": { "Name": "\[parameters('serviceTier')\]" } } } The name, location, and sku information is needed to provision a workspace, and their values are provided using parameters. The workspace after provisioning is shown in the following screenshot: ![2019 03 18 21 06 31 blog](/Blog/Images/2019-03-18_ 21_06_31-blog.png) Click on the OMS portal section to open the workspace portal. This portal is used to view all telemetry information captured by Operational Insights, configure Operational Insights, and provide dashboard features and functionality. The home screen of Operational Insights is shown here: ![2019 03 18 21 06 32 blog](/Blog/Images/2019-03-18_ 21_06_32-blog.png) The **Settings** section shows that four data sources are connected. These are four virtual machines connected to the OMS workspace from the test and production environments. A different strategy of having a separate workspace for each environment can be adopted, and it is left to readers to decide what's best for their applications and solutions. Operational Insights can be configured using the **Settings** tile: ![2019 03 18 21 06 33 blog](/Blog/Images/2019-03-18_ 21_06_33-blog.png) **OMS agents** You may have noticed that no assembly or code changes are made to the application to consume Operational Insights. Operational Insights depends on the installation of an agent on virtual machines. These agents keep collecting telemetry data from these hosts and send them to the Operational Insights workspace, where they are stored for a specified period of time depending upon the sku chosen. These agents can be installed manually on virtual machines. ARM virtual machine extensions install agents automatically, immediately after provisioning the virtual machines. The JSON code for provisioning of an agent on a virtual machine is shown here: { "apiVersion": "2015-06-15", "type": "Microsoft.Compute/virtualMachines/extensions", "name": "\[concat(variables('vmName'),copyIndex(1),'/omsscript')\]", "location": "\[resourceGroup().location\]", "dependsOn": \[ "\[concat('Microsoft.Compute/virtualMachines/',variables('vmName'),copyIndex(1))\]", "\[resourceId('Microsoft.Compute/virtualMachines/extensions', concat(variables('vmName'),copyIndex(1)),'powershellscript')\]" \], "copy": { "count": "\[parameters('countVMs')\]", "name": "omsloop" }, "properties": { "publisher": "Microsoft.EnterpriseCloud.Monitoring", "type": "MicrosoftMonitoringAgent", "typeHandlerVersion": "1.0", "settings": { "workspaceId": "\[parameters('WorkspaceID')\]" }, "protectedSettings": { "workspaceKey": "\[listKeys(variables('accountid'),'2015-11-01-preview').primarySharedKey\]" } } } The **workspaceId** and **accountid** are available from the **Settings** tile of the OMS workspace, and the copy element is used to deploy it to multiple virtual machines. This resource is a child resource of the virtual machine resource, ensuring that this extension is executed whenever a virtual machine is provisioned or updated. The configuration related to workspace ID and account ID is shown next. The primary key is used as an account ID for configuring the agents using ARM templates. ![2019 03 18 21 06 34 blog](/Blog/Images/2019-03-18_ 21_06_34-blog.png) **Search** The OMS workspace provides search capabilities to search for specific log entries, export all telemetry data to Excel and/or PowerBI, and search language specific to OMS. The **Log Search** screen is shown here: ![2019 03 18 21 06 35 blog](/Blog/Images/2019-03-18_ 21_06_35-blog.png) **Solutions** Solutions in OMS are additional capabilities that can be added to the workspace, capturing additional telemetry data that is not captured by default. When these solutions are added to the workspace, appropriate management packs are sent to all the agents connected to the workspace in the context of configuring themselves for capturing solution-specific data from virtual machines and containers, and then start sending it to the OMS workspace. The following screenshot shows the solution gallery and capacity and performance solution on the OMS workspace. Clicking on any solution and subsequently clicking on the **Add** button adds the solution to the workspace, as shown below: ![2019 03 18 21 06 36 blog](/Blog/Images/2019-03-18_ 21_06_36-blog.png) Azure provides lots of OMS solutions for tracking and monitoring different aspects of environments and applications. At a minimum, a set of solutions that are generic and applicable to almost any environment should be added to the workspace: * Capacity and performance * Agent health * Change tracking * Containers * Security and audit * Update management * Network performance monitoring **Alerts** Log Analytics allows us to generate alerts on the ingested data. It does so by running a pre-defined query composed of conditions on the incoming data. If it finds any records that fall within the ambit of the query, it generates an alert. Log Analytics provides a highly configurable environment for determining the conditions for generating alerts, time windows in which the query should return the records, time windows in which the query should be executed, and actions to be taken when the query returns results as alerts. ![2019 03 18 21 06 37 blog](/Blog/Images/2019-03-18_ 21_06_37-blog.png) The first step in configuring an alert is to create a saved search. A saved search is simply a search query against Log Analytics, as shown below: ![2019 03 18 21 06 38 blog](/Blog/Images/2019-03-18_ 21_06_38-blog.png) Save the query by providing a name for it. After saving the query, click on the **Alert** button from the **Log Search** menu. It provides the user an interface for defining and adding a new alert rule: ![2019 03 18 21 06 39 blog](/Blog/Images/2019-03-18_ 21_06_39-blog.png) Within this single screen, all configurations related to an alert rule can be performed. You need to provide a name, description, severity, and the query to be executed as part of the rule evaluation within their respective fields. The time window specifies the data interval in which the query should be executed. In the screenshot, whenever the rule is executed, it processes data from the last **15** minutes. The schedule section helps you configure the frequency of rule execution. It answers the question, **how frequently should the query run?** In the previous screenshot, the rule is executed every **15** minutes. The time window should be more than the alert frequency. The alerts can be further configured based on the number of results found. It need not be that an alert is generated for every instance of data found based on the query. It can further be quantified to accumulate a certain quantity of results before raising the requests. The alerts can also be generated based on metric measurements. Additional configuration to suppress alerts can be done. You can create a time interval that should elapse before the action is executed. In this case, alerts are generated but the action is executed only after the configured interval has elapsed. The **Actions**section allows us to configure things that should follow an alert. Generally, there should be a remedial and/or notification action. Log Analytics provides four different ways to create a new action. They can be combined in any way. An alert will execute any and all the following configured actions: * **Email notification**: This is the simplest, and sends an email to the configured recipients: ![2019 03 18 21 06 40 blog](/Blog/Images/2019-03-18_ 21_06_40-blog.png) * **Webhook:** A webhook executes an arbitrary external process using a HTTP POST mechanism. For example, a REST API can be executed, or the Service Manager/ServiceNow APIs can be invoked to create a ticket: ![2019 03 18 21 06 41 blog](/Blog/Images/2019-03-18_ 21_06_41-blog.png) * **Runbooks:** This action executes Azure Automation runbooks. * **ITSM actions:**ITSM solutions should be provisioned before using this option. It helps with connecting and sending information to ITSM systems. If you found this article interesting, you can explore [Azure for Architects](https://www.amazon.com/Azure-Architects-Implementing-containers-serverless-ebook/dp/B07NDBN6XM?utm_source=%20wsvincent&utm_medium=referral&utm_campaign=ThirdPartyPromotions) to create advanced data and integrated solutions using Azure Event Grid, functions, and containers. [Azure for Architects](https://www.packtpub.com/virtualization-and-cloud/azure-architects-second-edition?utm_source=%20wsvincent&utm_medium=referral&utm_campaign=ThirdPartyPromotions) can help you develop a full-fledged Azure cloud solution that is Enterprise class and future-ready. ### Frequently asked questions **Do I need to change my application code to use Log Analytics?** No. It depends on installing an agent on the virtual machines. The agents keep collecting telemetry from the hosts and send it to the workspace, so no assembly or code changes are made to the application. **What are OMS solutions?** Additional capabilities added to the workspace to capture telemetry that is not captured by default. Generic ones the post recommends include capacity and performance, agent health, change tracking, containers, security and audit, update management and network performance monitoring. **What actions can a Log Analytics alert trigger?** Four kinds, in any combination: an email notification, a webhook (an HTTP POST that can run a REST API or create a ServiceNow ticket), an Azure Automation runbook, and an ITSM action, which requires ITSM solutions to be provisioned first. --- ## Logging to AWS Cloudwatch using Nlog URL: https://www.ayobamiadewole.com/Blog/Logging-to-AWS-Cloudwatch-using-Nlog/ Category: Others Published: 2019-03-01 Summary: NLog can send .NET application logs to AWS CloudWatch Logs: add the NLog and AWS.Logger.NLog NuGet packages, configure a CloudWatch target in NLog.config, and supply AWS credentials, preferably from a credentials file rather than hard-coded keys. **Introduction** The importance of logging in enterprise software application development, cannot be overemphasized. Logging is the process of recording events, inner workings, and operations, warnings and errors of software applications. Logging libraries and frameworks can generate log files, database entries, queues messages which can be used to debug, trace and monitor enterprise applications in real time. Log entries generated can be used to troubleshoot, monitor, debug and resolve issues. Logging has also been referred to as the most powerful tool any software engineer can ever process - I totally agree with this. **NLog** Logging is a software development cross-cutting concern that has been made easier over the years in with the continuous introduction of new logging libraries and improvement of existing robust libraries available on Nuget. One of such libraries is NLog - a logging library that provides support for the different .Net platforms and gives the flexibility to write log entries to numerous targets which include file, console, database and AWS console. Also, NLog flexibility permits on the fly changing of logging configuration. Read more on NLog capabilities at [https://nlog-project.org](https://nlog-project.org) **AWS Console Logging** Using Amazon Cloudwatch Logs service, log entries generated by logging frameworks in .Net can now be written, monitored and accessed through AWS console. AWS Cloudwatch Logs service supports NLog and other .Net logging frameworks.console. In order to write logs to AWS cloudwatch, Nlog and AWS.Logger.NLog Nuget packages will be added to the .Net project. Afterwhich the NLog.config file generated when NLog library is added to a .Net project will have to be updated to use CloudWatch Logs as target. See the NLog configuration file below. Once the configuration file is set, a wrapper class can be created around the logging library where the AWS keys are set and the logging specific configurations can be initialized. See the code snippet for NLogWrapper class below. public class NLogWrapper { private readonly Logger logger; public NLogWrapper() { Configure(); } public void Configure() { var config = new LoggingConfiguration(); string accessKey = "put your access key here"; string secretKey = "put your secret key here"; var awsTarget = new AWSTarget() { LogGroup = "Log.ConfigPOC", Region = "us-east-1", Credentials = new Amazon.Runtime.BasicAWSCredentials(accessKey,secretKey) }; config.AddTarget("aws", awsTarget); config.LoggingRules.Add(new LoggingRule("\*", LogLevel.Debug, awsTarget)); LogManager.Configuration = config; } public void Info(object message) { logger.Info(message); } public void Error(Exception exception) { logger.Error(exception); } } A much cleaner approach without having to hard code the credentials will be to have the AWS credentials in a file placed in a location in the project or the application server. The path to the credential file will then be added to app.config or web.config file like below. Where **AWSProfileName** will have the same value as the heading contained in the credential file and **AWSProfilesLocation** will contain the path to the credential file. The content of the credential files will have the format below: \[test\] aws\_access\_key\_id = put key here aws\_secret\_access\_key = put key here The Nlog.config file will be updated with configuration entries like what is below. Also, the NLogWrapper class will be updated and the hard coded AWS credentials and configurations will be removed, the class will be modified to to what is in the snippet below. public class NLogWrapper { private readonly Logger logger; public NLogWrapper() { logger = LogManager.GetCurrentClassLogger(); } public void Info(object message) { logger.Info(message); } public void Error(Exception exception) { logger.Error(exception); } } This will begin to write the log entries to AWS console. The log entries can be reviewed from the AWS dashboard and metrics can be built and around the logs. ### Frequently asked questions **How do I send NLog logs to AWS CloudWatch?** Add the NLog and AWS.Logger.NLog NuGet packages to the .NET project, then update the NLog.config file to use CloudWatch Logs as a target. Log entries can then be reviewed in the AWS console. **Where should the AWS credentials go?** The post first hard-codes the access and secret keys in a wrapper class, then recommends a cleaner approach: keep the credentials in a file in the project or on the application server, and add AWSProfileName and AWSProfilesLocation keys to app.config or web.config. --- ## Image to PDF Conversion using Google Script URL: https://www.ayobamiadewole.com/Blog/Image-to-PDF-Conversion-using-Google-Script/ Category: Others Published: 2019-01-28 Summary: In Google Apps Script, converting an uploaded photo to PDF through a base64 HTML blob can distort phone-camera images because it ignores orientation. Inserting the image into a temporary Google Doc and exporting that as a PDF keeps it correct. **Introduction** There are diverse providers of services for collecting data from customers and users through forms and survey. However, Google form provides the flexibility of scripting and extending the data collection and form submission processes when users interact with the forms, surveys or opinion polls created using Google form. For most Google form creators, the default behaviour and functionalities provided out of the box can serve the data collection purposes. This is because Google makes it easy to access the data provided by users when they filled the form through response sheet attached to the form while additionally providing a response page for viewing the responses with pictorial and graphical representation of the responses. File of supported format and types can be added to a created form, making it easy to accepts files from users. When a form has file upload in it, Google saves the uploaded file in the Google drive of the Google account that owns the form and provides the accessible url in the response sheet, giving the flexibility to download the file using the provided url. While files of allowed format can be uploaded through the form’s file upload, Google doesn't provide an explicit way of converting files from one format to another, let's say from text or image to pdf during or after upload through the form. For example let's say we have a form that allows users to upload invoice files which can be a pdf file or an image taken with their mobile devices, like in the screen below. There might be a requirement that necessitate the conversion of the images to pdf and then send to an email address for processing. ![2019 01 27 21 06 08 blog](/Blog/Images/2019-01-27 21_06_08-blog.png) There is a well known approach which is to get the bytes of the uploaded image file, encode it into base64 and create an HTML image tag with the encoded bytes passed into the src attribute. After which Utilities class newBlob function is used to create an HTML blob from the content before finally getting a pdf blob from it, see the snippet below function getUploadedInvoice(fileId){ var uploadedFile = DriveApp.getFileById(fileId); var downloadUrl=uploadedFile.getDownloadUrl(); var mimeType=uploadedFile.getMimeType(); var filename=uploadedFile.getName(); var blob=null; var fileType=mimeType.toLowerCase(); if( fileType=="image/png" || fileType =="image/jpeg"){ var base64 = Utilities.base64Encode(uploadedFile.getBlob().getBytes()); var html = '![data:'+file Type+';base64,'+base64+'](data:'+fileType+';base64,'+base64+')'; blob = Utilities.newBlob(html, MimeType.HTML).setName(filename + ".pdf"); blob = blob.getAs(MimeType.PDF); } else if(fileType == "application/pdf"){ blob = uploadedFile.getAs('application/pdf'); } return blob; } This approach works and generates pdf file from the uploaded image. However, there is a limitation to it, when mobile users upload image files by directly taking the pictures using the phone camera, the above snippet doesn't handle the orientation of the image correctly, hence the final pdf has images distorted from the original look. ![2019 01 27 21 06 41 blog](/Blog/Images/2019-01-27 21_06_41-blog.png) This issue can be fixed by using a different approach which is to create on the fly a temporary Google document and insert the image into the body of the document, save it and generate a pdf blob from the created file. Using this approach, the orientation of the image is always intact without being distorted. The approach uses Drive Service for creating the temporary document which the image is inserted. ![2019 01 27 21 06 31 blog](/Blog/Images/2019-01-27 21_06_31-blog.png) function getUploadedInvoice(fileId){ var uploadedFile = DriveApp.getFileById(fileId); var fileType=uploadedFile.getMimeType().toLowerCase(); var blob=null; if( fileType=="image/png" || fileType =="image/jpeg"){ var image = uploadedFile.getBlob(); var filename=uploadedFile.getName(); var doc = DocumentApp.create(filename); var insertedImage = doc.getBody().insertImage(0, image); resizeImage(insertedImage); doc.saveAndClose(); blob = doc.getAs(MimeType.PDF); tempFileId=doc.getId(); } else if(fileType == "application/pdf"){ blob = uploadedFile.getAs(MimeType.PDF); } return blob; } function resizeImage(image) { var height = image.getHeight(); var width = image.getWidth(); var factor = height / 800; image.setHeight(height / factor); factor = width / 540; image.setWidth(width / factor); } ### Frequently asked questions **How do I convert an uploaded image to PDF in a Google Form with Apps Script?** The well-known approach gets the image bytes, base64-encodes them into an HTML image tag, creates a blob with Utilities.newBlob and gets a PDF blob from it. It works, but has a limitation with phone photos. **Why does the converted PDF look distorted?** When mobile users take a picture directly with the phone camera, that approach does not handle the image's orientation correctly, so the PDF shows the image distorted. **What is the fix?** Use the Drive service to create a temporary Google Document on the fly, insert the image into its body, save it and generate the PDF blob from that document. The orientation is always kept intact. --- ## The Journey from a Blogger to an Author URL: https://www.ayobamiadewole.com/Blog/From-a-Blogger-to-an-Author/ Category: C# Published: 2018-11-01 Summary: How the author went from blogging to writing "C# and .NET Core Test-Driven Development": a Packt editor emailed them in 2017 after reading their 2012 TDD post, writing began in October 2017, and the book was released on 18 May 2018. In the fall of 2015, I finally made up my mind to get a Master’s degree in Computer Science. It’s one of my numerous :) life goals that needed to be achieved at that time. After careful research and consideration, I opted for the option of combining full time work with schooling. This decision eventually turned out to be very rewarding but stressful and challenging. Upon completing the Master’s program in September 2017, I decided to take a break from everything :) and go on vacation to celebrate the little achievement. Just few days into the vacation, I got a mail from Packt Publishing Acquisition Editor, proposing to me the idea of writing a book. I am an Acquisition Editor at Packt. Currently I am looking for a potential author for a book titled C# and .NET Core Test-Driven Development. This book is for .NET developers who would like to build efficient applications by implementing principles of Test Driven development. I read your blog on this topic and I know you are an expert. I believe your contributions as an author will be greatly appreciated by the readers. If writing this book interests you, please get in touch. I'd be happy to share more details. Apparently, the editor had read an article on my blog about [Test Driven Development](http://ayobamiadewole.com/Blog/Test-Driven-Development\(TDD\)-in-Csharp) in C# that I wrote in January of 2012. It was an understatement to say I was super excited reading the mail, this is because I have always been very passionate about being an author. And to crown it, I was to write on Test Driven Development, a concept I have been enthusiastic about right from when I started enterprise software development. Series of emails were exchanged which necessitated reviewing the book brief, working on the outline and signing the book contract after a go ahead from Legal Adviser, I eventually started writing the book in October 2017. Setting out to write the book made me realize there is a clear difference from being a blogger and being an author. Just like blogging, when writing a book, you must choose a style of writing, be clear, concise and assertive. However, you must be ready to work with content editor, technical editor and reviewers, who will constructively edit and criticize the writeup. Fortunately for me the team of professionals and experts at Packt Publishing assisted me in the journey of transitioning from being a blogger to an author. In mid-May, 2018, after months of writing the book’s contents, editing and reviewing by the team of professionals at Packt, I finally woke up to this golden mail from the Acquisition Editor. Congratulations on the publication of C# and .NET Core Test-Driven Development! We have a fantastic end product with this book as a result of everyone's hard work and especially your dedication and patience. It was an honor working with a skilled professional like yourself and I hope we can work on another book in the future. If you would like to write another book or discuss any book project, feel free to get in touch with me. Again, many congratulations on this wonderful achievement! The book was eventually released to the public on May 18, as a perfect 2018 birthday gift. I have gotten positive feedback and reviews from readers who have bought and read the book. The book is available on [Amazon](https://www.amazon.com/C-7-Test-Driven-Development-ebook/dp/B0772S8R7Q). --- ## Increase Your Productivity With Pomodoro Technique URL: https://www.ayobamiadewole.com/Blog/Increase-your-productivity-with-pomodoro-technique/ Category: Others Published: 2017-10-09 Summary: The Pomodoro Technique is a time management method: plan your tasks, break them into pieces that fit in 25 minutes, work without interruption, and take breaks between them. The author manages tasks in Todoist and often works in 1-hour cycles. **Introduction** Lately, I had a conversation with a developer friend on how challenging it can be as a developer having to work on multiple projects concurrently, attend meetings and responds to urgent emails and still remain productive. I mentioned to him a technique I have adopted since I was in the university and still continue to use with great success - Pomodoro Technique. **Pomodoro Technique** Lately, I had a conversation with a developer friend on how challenging it can be as a developer having to work on multiple projects concurrently, attend meetings and responds to urgent emails and still remain productive. I mentioned to him a technique I have adopted since I was in the university and still continue to use with great success - Pomodoro Technique. Pomodoro technique is a time management methodology originally invented by Francesco Cirillo. He named it Pomodoro because of the shape of the timer clock (tomato shaped) that he used to keep track of his tasks. The technique involves planning at the beginning of the day what activities are to be undertaken and breaking down the tasks into smaller subtasks with assigned priority and the tasks should be completable in 25 minutes. You should record the tasks in a to do list app or on paper if desired. I normally use Todoist a to do list and task manager for creating and managing my tasks, it is multi-platform and allows having my tasks accessible on my computers and mobile, with ability. to before selecting the topmost task or task with highest priority and set. You need to have a timer to notify you of a completion of a pomodoro, todoist paid subscription has a timer that can notify you or alternatively you can use tomato timer at [https://tomato-timer.com/](https://tomato-timer.com/) To effectively use the pomodoro technique, you need to have breaks after each pomodoro or completion of a task which can be 25 minutes as recommended by the technique. A pomodoro should not be interrupted, but in reality there will be interruptions, these are to be recorded in the todoist app for considerations later, you have to resist the urge to stop the task you are currently working and quickly attend to the interruptions. I personally prefer and use a cycle of 1 hour after which I take 5 to 10 minutes breaks to maybe check my social media feeds, take a coffee or tea break to cool off, refresh and be active for another cycle. **Conclusion** Using pomodoro or any other time management technique is to effectively make you productive and get a lot done within a reasonable amount of time, for any technique to work you have to be consistent and disciplined to get desired result. ### Frequently asked questions **What is the Pomodoro Technique?** A time management methodology invented by Francesco Cirillo, named after the tomato-shaped timer he used. You plan the day's activities, split them into smaller prioritized subtasks that can each be completed in 25 minutes, and work through them one at a time. **What should I do about interruptions?** A pomodoro should not be interrupted. Record interruptions (the author logs them in Todoist) to consider later, and resist the urge to stop the current task to attend to them. **How long should breaks be?** Take a break after each pomodoro. The author personally prefers a cycle of one hour followed by a 5 to 10 minute break. --- ## Writing Large Datasets to Excel File using EPPlus in C# URL: https://www.ayobamiadewole.com/Blog/How-to-write-large-dataset-to-excel-file/ Category: C# Published: 2017-04-29 Summary: EPPlus handles very large Excel exports in C# where ClosedXML ran out of memory. In the author's test it saved 5 sheets of 1 million rows each in 61 seconds, using fewer lines of code than ClosedXML. **Introduction** Prior to the availability of [Open XML SDK](https://www.microsoft.com/en-us/download/details.aspx?id=30425) and other Excel libraries in . Net, manipulating Microsoft excel files from ASP.Net web applications was a daunting task. [Open XML SDK](https://www.microsoft.com/en-us/download/details.aspx?id=30425) makes it easy to manipulates documents that conforms to the Office Open XML file formats specifications with a trade-off of writing plenty lines of code. Similar to OpenXML, [ClosedXML](https://github.com/closedxml/) is another elegant library that further simplifies the process of reading and writing to excel files and hides the complexities involved, without need of dealing with XML documents. Recently, I had to resolve an issue of a corrupt report file of an enterprise application in production. Reviewing the logs indicated that the reporting module threw a System.OutOfMemory Exception which originated from ClosedXML. The excel file being created had 5 sheets with around 400,000 rows each and ClosedXML could not handle this large amount of data. **EPPlus** Apparently a lot of folks have had the same issue using the library and many suggested different workaround or alternative libraries. Catching my attention from the various suggestions is [EPPlus](http://epplus.codeplex.com) library which has been around for a while but never really had much publicity like OpenXML and ClosedXML. The library is available for use via Nugget. To test the performance and robustness of this library, I decided to throw a large dataset at it. I tried to make it create 5 sheets in a single workbook and save 1M rows in each sheet. It took 61 seconds on a DELL Latitude E5470 (Intel Core i5, 8GB RAM and 250GB SSD) for the library to process and save the records. It handled the large data graciously and also with fewer lines of codes compared to ClosedXML. See the source code below. using OfficeOpenXml; using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; namespace Epplus\_Poc { class Program { static void Main(string\[\] args) { List codeDetails = PopulateCodeDetails(); FileInfo fileInfo = new FileInfo(@"C:\\Temp\\file.xlsx"); using (ExcelPackage excelPackage = new ExcelPackage(fileInfo)) { var workSheet = GetWorkSheet(excelPackage, 0); var workSheet1 = GetWorkSheet(excelPackage,1); var workSheet2= GetWorkSheet(excelPackage,2); var workSheet3= GetWorkSheet(excelPackage,3); var workSheet4 = GetWorkSheet(excelPackage,4); workSheet.Cells\["B2"\].LoadFromCollection(codeDetails, false, OfficeOpenXml.Table.TableStyles.Medium1); workSheet1.Cells\["B2"\].LoadFromCollection(codeDetails, false, OfficeOpenXml.Table.TableStyles.Medium1); workSheet2.Cells\["B2"\].LoadFromCollection(codeDetails, false, OfficeOpenXml.Table.TableStyles.Medium1); workSheet3.Cells\["B2"\].LoadFromCollection(codeDetails, false, OfficeOpenXml.Table.TableStyles.Medium1); workSheet4.Cells\["B2"\].LoadFromCollection(codeDetails, false, OfficeOpenXml.Table.TableStyles.Medium1); excelPackage.Save(); } } static ExcelWorksheet GetWorkSheet(ExcelPackage excelPackage, int count) { var workSheet = excelPackage.Workbook.Worksheets.Add("Content - "+count); workSheet.View.ShowGridLines = false; workSheet.Cells\["B1"\].Value = "Code"; workSheet.Cells\["C1"\].Value = "Time"; workSheet.Cells\["D1"\].Value = "Date"; workSheet.Cells\["B1:D1"\].Style.Font.Bold = true; return workSheet; } public static List PopulateCodeDetails() { List codeDetails = new List(); Random random = new Random(); for(int i=1; i<=1000000; i++) { CodeDetail codeDetail = new CodeDetail(); codeDetail.Code = random.Next(12324343).ToString(); codeDetail.Time = DateTime.Now.ToShortTimeString(); codeDetail.Date = DateTime.Now.ToShortDateString(); codeDetails.Add(codeDetail); } return codeDetails; } } public class CodeDetail { public string Code { get; set; } public string Time { get; set; } public string Date { get; set; } } } When next you need to process large dataset and save it to Excel file, you can try EPPlus library. ### Frequently asked questions **Why did ClosedXML fail on a large report?** In the author's production case the reporting module threw a System.OutOfMemoryException that originated from ClosedXML. The workbook had 5 sheets of around 400,000 rows each, and the report file ended up corrupt. **How well does EPPlus handle large datasets?** In the author's test on a Dell Latitude E5470 (Core i5, 8GB RAM, SSD), EPPlus created one workbook with 5 sheets of 1 million rows each and saved it in 61 seconds, with fewer lines of code than ClosedXML. --- ## Pdf Conversion to Image Using MagickNet in C# URL: https://www.ayobamiadewole.com/Blog/Pdf-Conversion-to-Image-Using-MagickNet-in-Csharp/ Category: C# Published: 2016-12-01 Summary: Magick.NET, a .NET wrapper around ImageMagick, can convert every page of a PDF to images in C# as long as Ghostscript is installed on the server. Point MagickNET at the Ghostscript path and set a temp folder. **Introduction** Sometimes back, I worked on an asp.net MVC application which had a feature request to support conversion of pdf documents to images to allow viewing of the documents on devices that have Javascript disabled and most likely do not have native pdf reader app installed. Initially, I thought of developing a custom library for this, but could not due to tight deadline of the project. I decided to search Nugget Gallery for hosted libraries that can be used for this purpose, but most of the libraries available on Nugget are limited in functionalities and not usable. Searching [codeplex.com](https://www.codeplex.com), I came across Magick.net, [https://magick.codeplex.com](https://magick.codeplex.com) an image processing library built as a wrapper on ImageMagick which can also convert pdf files to images if Ghostscript is installed on the application server. Ghostscript can be downloaded at [http://www.ghostscript.com/download/gsdnld.html](http://www.ghostscript.com/download/gsdnld.html) The PdfToImageConverter class I wrote for the pdf to image conversion is explained below. Once Ghostscript is installed, MagickNET needs to be aware of the installation path and also a Temp folder needs to be set as it writes some temporary files to disk. The library can read pdf file of any size and convert all the pages in the file to images and save the converted images to the path specified in the method parameter. See the source listing of the PdfToImageConverter class below. public class PdfToImageConverter { public PdfToImageConverter(string filePath) { MagickNET.SetGhostscriptDirectory(filePath+"Lib"); MagickNET.SetTempDirectory(filePath + "\\Temp"); } public int ConvertFileToImages(string filePath, string destinationPath) { int noofPages = 0; MagickReadSettings magickReadSettings = new MagickReadSettings(); magickReadSettings.Density = new Density(300, 300); using (MagickImageCollection magickImageCollection = new MagickImageCollection()) { magickImageCollection.Read(filePath, magickReadSettings); int page = 1; noofPages = magickImageCollection.Count; foreach (MagickImage magickImage in magickImageCollection) { magickImage.Format = MagickFormat.Png; string imageFilePath = string.Concat(destinationPath,"file-", page, ".png"); magickImage.Write(imageFilePath); page++; } } return noofPages; } } ### Frequently asked questions **How do I convert a PDF to images in C#?** Use Magick.NET, an image processing library built as a wrapper on ImageMagick. It can read PDF files of any size and convert all their pages to images saved to a path you specify, provided Ghostscript is installed on the application server. **What setup does Magick.NET need to handle PDFs?** Once Ghostscript is installed, MagickNET has to be told its installation path, and a temp folder must be set because the library writes temporary files to disk. **Why convert PDFs to images at all?** The author needed it so documents could be viewed on devices that have JavaScript disabled and likely have no native PDF reader app installed. --- ## Importing Excel File with SheetJs in Asp.net MVC URL: https://www.ayobamiadewole.com/Blog/Importing-Excel-File-with-SheetJs/ Category: Asp.Net Published: 2016-02-19 Summary: SheetJS parses Excel files in the browser and converts them to JSON or CSV, so a bulk-import feature can post the data to the server by Ajax without server-side Excel parsing. The ASP.NET MVC example deserializes the JSON with Json.NET. **Introduction** I recently worked on a project with a feature requirement to allow bulk import of data from Microsoft Excel files. My choice of framework to use would have been either [Open XML SDK](https://www.microsoft.com/en-us/download/details.aspx?id=30425) or [LinqToExcel](https://github.com/paulyoder/LinqToExcel) because I have used both libraries in the various projects in the past and they performed well. But common we are in 2016, I wanted to try out a new library. I asked a programmer friend, [Ahmed Opeyemi](http://ahmedopeyemi.com) if he has used any library lately and he recommended [SheetJs.](https://github.com/SheetJS/js-xlsx) I decided to do some check on the library to see if it will meet my requirement. **SheetJS** After reading the library’s online documentation on [Github](https://github.com/SheetJS/js-xlsx), I like the fact that it can convert the data extracted from the excel file to CSV or JSON or even generate a list of formulae from the excel file being parsed. The generated data that can later be uploaded to the server by an Ajax Post. Also, since the extraction and JSON conversion is done directly on the user's browser it frees the server of the processing that hitherto would have been done on it. **Building the application** Navigate to [https://github.com/SheetJS/js-xlsx](https://github.com/SheetJS/js-xlsx) to download the library and include the xlsx.core js library in the view where you want the data upload feature I decided to handle the onchange event of the html file input so that whenever the user selects the excel file, the data contained therein can be extracted and posted to the server through. The code snippet for performing the excel file parsing and uploading it to the server is as shown below The portion of codes below does the magic of extracting the data from the excel file. var workbook = XLSX.read(data, { type: 'binary' }); var result = {}; workbook.SheetNames.forEach(function (sheetName) { var roa = X.utils.sheet\_to\_row\_object\_array(workbook.Sheets\[sheetName\]); if (roa.length > 0) { result\[sheetName\] = roa; } }); And to deserialize the JSON data to a list of .net object , I use the popular [Json.NET](http://www.newtonsoft.com/json) library. The code of the ActionResult method in the Controller class is below \[HttpPost\] public ActionResult DataUpload(FormCollection formCollection) { string dataToUpload = formCollection\["dataToUpload"\]; int recordsUploaded = 0; GroupDTO groupDTO = JsonConvert.DeserializeObject(dataToUpload); // do something with the data return Json(new { records = string.Concat(“Total records”, recordsUploaded)}); } The GroupDTO class definition is below public class GroupDTO { public List Data { get; set; } public long ChurchId { get; set; } } public class GroupJson { public string Name { get; set; } public string Description { get; set; } } And that is it, you have the data extracted and uploaded with little work at the server end. ### Frequently asked questions **Why parse Excel files in the browser with SheetJS?** The extraction and JSON conversion happen in the user's browser, which frees the server of that processing. SheetJS can also output CSV or a list of formulae from the file. **How does the upload work in the ASP.NET MVC example?** The page handles the file input's onchange event, extracts the data from the chosen Excel file as JSON in the browser, and posts it to a controller action by Ajax, where Json.NET deserializes it into a list of .NET objects. --- ## How to Provision Cisco IP Phones 8861, 7861,7821,7811 and 7942G for use with Freeswitch PBX URL: https://www.ayobamiadewole.com/Blog/How-to-provision-Cisco-IP-Phones-for-use-with-Freeswitch/ Category: VoIP Published: 2015-11-16 Summary: Cisco IP phones ship configured for Cisco Unified Communications Manager (SCCP), so using them with FreeSWITCH means upgrading their firmware to SIP from a TFTP server, with XML config files. The post covers the 8861, 7861, 7821, 7811 and 7942G. **Introduction** Using soft phones with Freeswitch PBX allows the utilization of existing PCs or smart phones as the communicating devices. But often there are times when the requirements for setting up or deploying a Unified Communication solution necessitate the use of IP phones. Freeswitch has been tested and works well with various IP phones, more details can be found at [https://freeswitch.org/confluence/display/FREESWITCH/Phones](https://freeswitch.org/confluence/display/FREESWITCH/Phones) In this blog post I will explain how I got five different models of Cisco IP phones provisioned for use with Freeswitch. Cisco IP phones are factory configured or provisioned to work only with Cisco Unified Communication manager which mainly uses SCCP as communication protocol. Freeswitch supports SCCP though, but to enable the IP phones work seamless with Freeswitch, the firmwares on the various phones need to be upgraded to enable them use SIP and subsequently provisioned while doing the upgrade. **Required Tools** Cisco IP phones usually will get their firmware upgrade from a TFTP server residing on the network. Newer models of Cisco phones allow manually inputing IP address of the TFTP server in the phone configuration. But for the old model; 7942G, the DHCP server will have to advertise the address of the TFTP server to it. **On a Windows PC** Tftpd32 is an utilitiy app that will make aWindows PC serve as both the DHCP and TFTP server at the same time, it can be downloaded from. **On a Linux PC** I did the provisioning 8861, 7861,7821,7811 models on a Debian PC. First install the tftp application running this command on the terminal. apt-get install atftpd Then run vi /etc/default/atftpd to edit the content of atftpd by adding – port 69 to the content of the file to make it look like what is shown below USE\_INETD=false OPTIONS="--port 69 --tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /srv/tftp" Save the file and exit vi. Start the tftp server by running this command /etc/init.d/atftpd start To allow connection to be made to the tftp server on port 69 run the command below, while substituting the IP for your network IP iptables -A INPUT -p udp -s 172.16.20.0/24 -m multiport --dport 69 -m state --state NEW -j ACCEPT I recommend downloading firmware files for the phones directly from Cisco software download center, you can register an account which will be verified and will allow you to download the SIP upgrade files. After downloading the files, you should extract and copy the contents to the tftp server root folder. **Firmware Upgrade and Provisioning** Before the upgrade can be done, some required xml files need to be created and placed in the tftp server root folder. **Diaplan.xml**