I always thought of logging page hits as complimentary to
Google Analytics. By adding a SessionId cookie which expires when the browsing session ends and
ASP.NET Core SignalR for monitoring active connections (online users), I have gotten closer to a Google
Analytics alternative. This series will explore some of the issues and considerations for collecting and analyzing
The project has been upgraded to ASP.NET Core 6.0 with Visual Studio 2022 from the ASP.NET Core
5.0 project. I migrated the Startup.cs to top-level statements in Program.cs. I also implemented global
usings and file scoped namespaces. I have enabled the nullable context and mitigated all warnings
and issues. I have updated the project with Twitter Bootstrap v5. See
ASP.NET Core 5.0 - Migrate To Bootstrap v5. This project implements
Bootstrap Native. The article series continues the ASP.NET Core 5.0 series with an introduction to the
ASP.NET Core 6.0 - Homegrown Analytics Project. Version 2 implements a new schema and UI.
The Analytics middleware collects and analyzes the properties of the HttpContext and Request. The
middleware creates and stores a new Session if the SessionId cookie is not found. The middleware
looks for a known UserAgent and creates and stores a new UserAgent if not found. The middleware
creates and stores a new PageHit and updates the NotFound property on 404 Response status.
PageHit pageHit = new(method, path, queryString, refererHeader, userAgentId, sessionId, anonymizedIp, cookieConsent, screenSize, Convert.ToInt32(minutesOffset), userId);
After implementing and updating analytics with my live sites multiple times, I have refactored
the HAP V2 with an updated analytics schema and UI. The integration process is simplified with
an Analytics area folder. After the AnalyticsDbContext is configured, you can use the Package
Manager Console to to execute "update-database -Context AnalyticsDbContext" to apply the
analytics schema migration. The CreateDatabaseAndAnalyticsSchema.sql and
CreateAnalyticsSchema.sql scripts are located in the Areas/ Analytics/ Data/ SqlScripts folder.
The PurgeExpiredAnalytics script deletes expired related records. The RemoveAnalyticsSchema
will remove all analytics and Hangfire data and schema. The EnsureAnalyticsDefaultData
HostedService sets the AnalyticsSchema version and creates the required default
UserAgent and Session. The
KenHaggerty. Com. SingleUser. Net6 NuGet package segregates the user logic. See
ASP.NET Core 6.0 - Analytics Schema and Integration.
The HAP implements a cookie consent banner and CheckConsentNeeded from Microsoft.
AspNetCore. CookiePolicy to request user consent for non-essential cookies. If granted, I use
page with a revoke cookie consent function. I consider the SessionId cookie which expires when
the browsing session ends as essential. The middleware collects and analyzes the properties of
the HttpContext and Request. The middleware creates and stores a new Session if the
SessionId cookie is not found. The middleware looks for a known UserAgent and creates and
stores a new UserAgent if not found. The middleware creates and stores a new PageHit and
updates the NotFound property on 404 Response status. If cookie consent is granted, the screen
size and GMT time offset are recorded. I capture the HttpContext. Connection. RemoteIpAddress
but I make it anonymous by blanking the Host ID (the last segment) with zero. See
How-To Geek - How Do IP Addresses Work?.
The analytics schema includes foreign keys which allow Fluent API queries on related entities.
I use a nullable foreign key on AppUserId which allows anonymous and authenticated sessions
and page hits. Finding accurate information on creating indexes, foreign keys, and nullable
foreign keys is difficult. Deleting related records must be carefully implemented and tested.
If a user deletes themselves, the AppUserId on Session and PageHit must be set to null
before the AppUser can be deleted.
I added filters to chronicle lists of sessions and page hits but this did not represent trends over time.
HAP V1 had a client screen size page which displayed a bubble chart implemented with
D3 - Data-Driven Documents.
HAP V2 implements Chart.js.
Dashboard to display a line chart of the number of PageHits over time. The time period is selected
from Segments: 24 Hours, 7 Days, 5 Weeks, 6 Months, and 4 Quarters. When data history
allows, a data series of the previous segment is automictically added to the line chart. HAP V2
implements a svg world map from amCharts - Free SVG Maps,
NuGet - MaxMind.Db, and
IP to Country Lite database to highlight the map with the top
10 countries with the most page hits. The optional _AnalyticsLayout .cshtml with a 1900px
wide display container has been updated for easier integration.
The UserAgent has a Preferred property. I developed a UserAgent chart page which displays
the UserAgent's PageHits count over time and provides the functions to add or remove the
preferred status. Rather than trying to detect robot page hits, I can filter on Preferred UserAgents.
Homegrown Tag Helpers
Filtering a boolean property with a checkbox facilitates 2 states: include and only. I developed
a group of radio buttons to allow 3 states: include, exclude, and only. I use this boolean filter
on many pages for a lot of properties. Creating a custom TagHelper is easier than initial research
suggests. After implementing the BoolFilterTagHelper, I developed SegmentFilterTagHelper,
DaysFilterTagHelper, and CurrentIpFilterTagHelper. The DatalistSelectTagHelper generates
a select dropdown with an auto complete feature.
Client End Of Day
The data is stored with a created DateTimeOffset set to the UTC date and time. When the
TimeOffset cookie is not found, the charts use the GMT end of day. With the TimeOffset, the
charts and Segments represent the client's end of day. To accurately calculate the end of week,
month, and quarter you need to start with an accurate client's end of day. My hosting server is in
the Pacific time zone and I develop in the Eastern time zone. I had to publish and test multiple
times before getting this right.
I use Google Analytic's Active Users feature mainly to detect if any users are online before I take
the site offline for updates. I implemented and deployed an online user feature based on a couple
of examples I found which count online users with SignalR. AnalyticsSettings.cs has an option to
enable SignalR and the online user feature. The option is disabled by default.
Before I implemented Google reCAPTCHA v3 on the contact and register pages, I was getting
emails and users created by robots. Since, I haven't noticed any. You must register your site to
receive an API key and secret. You set the secret, key, and a pass/fail threshold in
AnalyticsSettings.cs. reCAPTCHA v3 returns a score (1.0 is very likely a good interaction, 0.0 is
very likely a bot). AnalyticsSettings.cs has an option to enable Google reCAPTCHA. The option
is disabled by default.
Online log maintenance like purging old records are usually long running tasks. Hangfire is
better than Background tasks for fire and forget tasks. Hangfire is easy to implement. The
Hangfire dashboard provides detailed status on running, completed, and failed jobs.
AnalyticsSettings.cs has an option to enable Hangfire. The option is disabled by default.