Report Generator

Overview

The Report Generator is used to generate reports containing data related to campaigns, groups, targets, and school. The type(s) of data that's included in a report depends on the type of report selected (see the Select Report section of this article). These reports are viewable on the PhishingBox portal and can also be downloaded in PDF format. 

To get started, navigate to Reports > Report Generator.

Use the links below to jump to a section:

1. Select Report
2. Cover Page Options
3. Report Sections
4. Filters
5. Custom Fields
6. Collate
7. Post-Report Generation Options
8. A Note About Flattened Data

Select Report

Pick the report you want to generate by using the "Select Report" drop-down. Reports in this list are sectioned off based on type. The various report types are described below.

NOTE: All report types can be customized, the user may enable or disable report sections as desired.

The following are the available report types:

NOTE: Review Report Sections to get more details about the available sections for each report type. 

• By Cadence Campaign - Select specific cadence campaign for reporting. 
  • Summary Report - This report includes a campaign summary, phishing email cadence pass/fail percentages, phishing template pass/fail percentages, phishing email cadence action breakdown, worst action percentages. If a training test is included in the cadence campaign, you will all get a training campaign summary, phishing & training side-by-side cadence comparison, training email cadence opened/unopened percentages, training template opened/unopened percentages, and training email cadence action breakdown. 
• By Campaign - Select specific campaigns for reporting. Reports by campaign will only allow one campaign per report.
  • Summary Report - The report includes a campaign summary, campaign historical data, campaign pass/fail percentages, and metric groups.
  • Summary Report w/ Executive Reports - Includes the Summary Report along with the By Test Executive Report(s) (see Executive Report description below).
• By Course - Generate reports for selected courses.
  • Enrollment Report - Course reports include course name, course description, enrollees, enrollment date, due date, start date, complete date, certificate, progress, and score.
• By Course Program - Generate reports for selected programs.
  • Program Enrollment Report - Program reports include program name, program description, enrollees, enrollment date, due date, start date, complete date, certificate, and progress.
• By Group - Generate reports for selected groups.
  • Summary Report - A quick summary of a group's phishing testing. Includes the group details, failure activity breakdown pie chart, last 12 phishing tests historical data, and last 12 tests pass/fail historical data.
  • Summary Report w/ Actions - Includes the group summary report along with the groups action breakdown table. (Will only print if there are less than 2500 targets plus actions in the group due to browser constraints).
  • Summary Report w/ Metrics - Includes the group summary report along with the group's historical test data, historical training test data, target phishing metric data, target training metric data, phishing template metric data, training template metric data, and a group course enrollment summary. (Will only print if there are less than 2500 targets in the group due to browser constraints).
  • Summary Report w/ Executive Reports - Includes the Summary Report along with test Executive Reports for all tests on the group.
  • Summary By-Date Report - The Summary Report with data filtered by a date range.
  • Date Comparison Report - A report that includes comparison data between two selected date ranges. The sections included are number of phishing messages sent during time period, number of unique phishing targets tested during time period, number of training messages sent during time period, and number of unique training targets tested during time period. 
  • Repeat Offender Report - A report showing targets in a group who have failed multiple times for the specified date range.
• By Target - Generate reports for selected targets. Search for targets by email in the "Targets:" field. Selected targets will appear in the field below "Targets:".
  • Individual Report - Includes the target's phishing summary, phishing test actions breakdown, training summary, and training test actions breakdown. Each target selected will generate an additional target summary and a target course summary.
  • Individual By-Date Report - Identical to the Individual Report, but you can select a date range to filter the included data.
• By Template - Report metrics for an account's templates. Useful for comparing template effectiveness on a template-by-template basis. 
  • Summary Report - Includes the number of tests in which the template has been included, the number of times the template has been sent as a phishing email, count of opens, open percentage, count of fails, failure percentage, count of reports, and report percentage.
• By Test - Generate reports for a single test or a group of tests. Test "Executive Reports" can be included in several other report types. Reporting by test allows you to filter test metrics based on target details including but not limited to department, city, and state. When you select a By Test type report, you'll be able to generate the report for phishing tests as well as training tests.
  • Executive Report - A simple test report. This includes the test details, the test's Net Reporter Score, a quick test summary that includes the number of messages sent and failures at a glance, passed vs. failed pie chart, most severe action pie chart, test results, received training pie chart, and failed received training pie chart.
  • Full Report - An Executive Report that includes a test comparison summary if multiple tests are selected, and an auto-enroll course summary if the selected tests configured course auto-enroll.
  • Exhaustive Report - A Full Report that includes a test actions breakdown, phishing IP address user hit locations and template information. The actions breakdown will only be included if there are fewer than 2500 actions on a test due to browser constraints.
  • Failed Only Report - Includes a failed-only test summary, test comparison summary if multiple tests are selected, actions breakdown for tests with less than 2500 actions, and phishing IP address user hit locations. Failed only reports filter out all actions that are not failures from reported metrics.
  • Actions Report - Includes the test's actions breakdown (a table containing a list of actions with their date, time, action type, IP address, browser, and OS) and phishing IP address user hit locations.

Back to top

Cover Page Options

These options allow you to make changes to the PDF cover page when you generate the report.

• Report Title - The report title is generated by default based upon the report type and data. However, if you wish to enter your own title you can do so here. This title will appear on the cover page and file name.
• Cover Page Image - Upload an image that will display as a cover page header. The default cover image can be configured in Branding Settings. 
• Company Name - Insert a company name that will be displayed on the report's cover page.
• Custom Cover Page Text - This is custom text that will appear on the cover page of your report when it is generated as a PDF. You can use paragraph, break, bold, and italic HTML tags. All other tags will be stripped.
• Prepared By - The name of the user who generates the report.
• Contact Email - The email address of the user who generates the report.
• Hide the date generated - By default, the date and time the report is generated will appear on the cover page of the PDF. However, if you wish to hide this date check this box.
• Cover Page Logo - A cover page company logo. The default logo can be configured in Branding Settings.

Back to top

Report Sections

Report sections can be enabled or disabled at will. Different types of reports will contain different reporting sections. To see which sections are included in our default reports, review the Select Report section of this article.

The 'Phishing Term Appendix' is included in all reports by default. The appendix lists any phishing specific terms that may be useful for interpreting the report data. Different phishing templates will generate different terms. Not all phishing templates will hit the same endpoints or lead to the same conclusions. For example, some phishing emails require report-by actions, some will lead to hook URLs. Every campaign is different and will generate different phishing terms. For more information, see our Phishing Term Appendix.

The 'Hide Annotations Section' switch will enable or disable the annotations sections of reports. Annotations provide additional metadata related to the report, including whether phish domains were flagged by search providers while a test was ongoing.

There will be many optional sections to include in each report type. The sections below will breakdown the reports by their groups and describe the available section toggles.

By Cadence Campaign

• Tests Action Summary (Only included if less than 2500 rows) - Includes action status percentages for each email and a list of actions from the campaign.

By Campaign 

• Campaign Summary - Includes campaign details, net reporter score, campaign historical data, and campaign pass/fail breakdown by test. 
• Include Test Executive Reports - This section provides a summary, net reporter score, and metrics for each test in the campaign. 
• Include Reported Emails Section - Provides total emails sent, total emails reported, and percent of emails reported. 
• Include Users Reported Phishing Table - Provides a table of targets who reported the email. The table includes the template name, date email sent, and time reported. 

By Course 

• Course Summary - Provides the course name & description, ID, created date, updated date, and type. It also includes an enrollment table that contains student name, student email, active groups the student is in, enrolled date, due date, started date, completed date, received certificate, progress, and score.
• Overall Student Summary - This section gives an overview of all the students enrolled in the selected courses. The data is aggregated by student, where it shows the total number of students enrolled in the selected course(s), total number of students that have not started any coursework and are not overdue in any coursework, total number of students that have at least one in-progress course and are not overdue in any coursework, total number of students that have completed and passed all assigned coursework, and number of students with at least one overdue course.
• Overall Enrollment Summary - This section gives an overview of the enrollment statuses for all selected courses. The data included is total enrollments, total enrollments not started, total enrollments in progress, total enrollments completed, and total enrollments overdue.

By Course Program 

• Course Program Summary - Provides the program name & description, ID, created date, updated date, and type. It also includes an enrollment table that contains student name, student email, active groups the student is in, enrolled date, due date, started date, completed date, received certificate, and progress.

By Group 

• Group Summary - This section gives an overview of data from all tests the group(s) are included in. The data includes a failure activity pie chart which is broke down into failure percentages of all failure actions. Here if a user clicks a link 3 times for example, all 3 clicks will be counted. Last 12 phishing tests historical data and last 12 phishing tests pass/fail percentages are included. 
• Group Historical Test Data - This gives a historical summary of all the data for each of the tests the group(s) are included in. The data included is test name, start date, target count, fail percentage, net reporter score, delivery status counts, and unique action type counts. 
• Group Target Metric Data (Only prints if under 2500 rows) - This section gives a summary of phishing and training metric data for each target in the group. The metric data includes target name, email, target status, group level status, tests count, delivery status counts, and unique action type counts. 
• Group Actions Breakdown (Only prints if under 2500 rows) - This section lists all the actions for the group in a tabular format. The data included is test name, action date, action type, filters, human interaction, and status. 
• Group Template Metric Data - This section provides metric data for each template sent to the group. The data included is test name, tests count, delivered count, open percentage, failure percentage, and reported count.  
• Group Historical data by Month - This section provides a line graph of target action data by month, and a line graph for target pass/fail percentage by month. 
• Group Test Comparison Summary (Only included if more than one test is included) - Gives a breakdown of each of the tests for the group for comparison. The data included in the breakdown is test name, started date, length of test, count of targets, failure percentage, net reporter score, templates, and unique action type counts. This section also provides average fail rate graphs and most severe action for all tests graph.
• Group Test Comparison Targets Summary (Only included if more than one test is selected) - Gives a breakdown of each of the tests for the groups' targets for comparison.  The data included is test name, test type, start date, end date, worst action, and pass/fail status. 
• Include Test Executive Reports - Provides Executive Reports for each test the group was included in. 
• Group Course Summary - Provides a list of enrollments for each target in the group. The data included is course name, date enrolled, date due, date started, date completed, progress, and score. 

By Group Repeat Offender 

• Group Summary - This section provides the group's created date, last updated date, service type, auto sync status, smart sync stat, and active target count.
• Repeat Offender Counts - This section breaks down the targets who have failed by the number of failures for the target. 
• List of Repeat Offender Counts - This section lists each target who is a repeat offender. The data included in this list is email address, name, failure count, last failed test, email click count, and data extended count. 
• Repeat Offender Breakdown by Timezones - Generates a list of each timezone that has repeat offender(s). The list will include the timezone name and repeat offender target count. 
• Repeat Offender Breakdown by Countries - Generates a list of each country that has repeat offender(s). The list will include the country name and repeat offender target count. 
• Repeat Offender Breakdown by Languages - Generates a list of each language that has repeat offender(s). The list will include the language name and repeat offender target count. 
• Repeat Offender Breakdown by Departments - Generates a list of each department that has repeat offender(s). The list will include the department name and repeat offender target count. 
• Repeat Offender Breakdown by Titles - Generates a list of each title that has repeat offender(s). The list will include the title name and repeat offender target count. 
• Repeat Offender Breakdown by Managers - Generates a list of each manager that has repeat offender(s). The list will include the manager name and repeat offender target count. 

By Target 

• Target Summary - This section breaks down the targets data into a summary including a pass vs fail comparison, most severe actions chart, target data by test, action table for each target, and the targets IP map and summary (An example IP map is shown below). 

• Target Course Summary - This section provides a list of enrollments for the selected target(s). The data included in this list is course name, date enrolled, date due, date started, date completed, and score. Only use this option if the report contains 25 or fewer targets. 

By Template

• Template Summary - This section provides a list of templates with their name, tests count, emails count, opens count, open percentage, failure count, fail percentage, reported count, and reported percentage. 

By Test 

• Test Summary - Includes test details, net reporter score, test metrics, test pass/fail pie chart, most sever actions pie chart, received training pie chart, and received training (failed only) pie chart. 

  NOTE: Received training refers to the target completing the training included in a training page. This does NOT refer to completing a School Course. 

• Test Comparison Summary - This section provides a pie chart for included tests average passed vs failed, most server action by user for all tests pie chart, test breakdown, and a repeat offenders list. The test breakdown includes the test name, start date, test length, email count, target count, failure count, failed percentage, group, and template(s) used.  
• Test Comparison Targets Summary - This section provides a list of targets with a comparison of the selected test(s). The data included in this list is test name, start date, end date, worst action, and status (pass/fail).  
• Click Rate by X - These sections will break down the targets click rate by the filtered selection.
• Test Action Summary (Only included if less than 2500 rows) - This section will display the targets test actions in a tabular format, an IP map, and an IP summary. 
• Test Training Summary - Includes auto enrollment configuration for each selected test.
• Auto-Enroll Summary - This section lists all auto-enrollments that were created as a result from a target failing in the selected test(s). 
• Test Template Details - This section provides the setup details for each template included in the test.
• Include Reported Emails Section - This section provides a pie chart containing reported/not reported percentages. 
• Include Users Reported Phishing Table - This section provides the targets who reported the email. The data included in this section is target name, template name, date email sent, and time reported.
  
• Test Action All - This section breaks down the actions for each target in the test.

Back to top

Filters

There are a variety of filtering options allow you to pull data according to your needs.

NOTE: Some filters will be hidden for specific reports. This means those filters are not applicable to that report. 

Action Filters

This section allows you to include/exclude certain information related to the actions performed by your targets. 

• Show Filtered Actions - Allows you to display the actions filtered out (not counted) due to IP filters or AHD. Displaying these actions does not cause them to be counted.
• Show Suspicious Actions - Allows you to display the suspicious actions (i.e., actions neither verified nor filtered). Displaying these actions does not cause them to be counted.
• Show IP & User Agent Data - Allows you to turn on/off the IP address, the coordinate data, and user agent information for the user’s actions.
• Show IP Map - Allows you to turn on/off the map showing where the various IP addresses are geographically located.

NOTE: See our Advanced Human Detection (AHD) article for more details on action types and statuses. 

Target Display Filters

This section allows you to include/exclude certain information related to targets.

• Show Failed Targets/Data Only - Only pulls data from tests where the target(s) failed.
• Show Targets with no actions on the Actions Breakdown - Turning this on will include targets with no test data on the Action Breakdown section.
• Redact Target Email Addresses - Allows you to hide the email addresses of your targets in the report. For security reasons, this is advised for sending via email.
• Redact Target Names - Allows you to hide the names of your targets in the report. For security reasons, this is advised for sending via email.

Managers

When generating Course Enrollment Reports, you have a couple filtering options regarding managers.

• Include Target Manager - Enabling this option will include the Target Manager column in the course enrollment summary. 
• Include Enrollment Manager - Enabling this option will include the Enrollment Manager in the course enrollment summary. 

Target Query Filters

Target query filters allow you to customize the data pulling queries to include/exclude matching your criteria. You can choose from any of the pre-defined target fields and any custom fields you have created. For example, if you only wanted to show actions for targets in the engineering department, then you'd enter a target query filter like this:

NOTE: The "Active" filter - requires you to enter either 1 (to filter by active targets) or 0 (to filter by inactive targets).

Groups

The Course Enrollment and Program Enrollment reports can be filtered by groups. 

Action Query Filters

The action query filters drop-down lets you filter the actions breakdown table by action type (actions of certain types can be included/excluded from the table).

Order By

The order by drop-down lets you to order tables (Actions Breakdown table, enrollments table, Target Metric Data table, etc.) ascending/descending by target fields.

Test Filters

• Hide Test with No Actions - Allows you to hide tests that have no actions from displayed data. This only affects the data displayed, not the analysis data.

Custom Fields

This drop-down allows you to add any of your custom field data to the target rows in the report. For spacing reasons, it is advised not to add more than 5 fields. The custom fields will only appear on the actions table.

Back to top

Collate

The Collate option becomes visible when running By Campaign > Summary Report w/ Executive Reports. This option will allow you to download a zip file containing the campaign's test executive reports in separate PDF files. The Save PDF button will become a Save Zip button, and the Email PDF button will become an Email Zip button.

Back to top

Post-Report Generation Options

Once your report has been generated the top of the screen will contain some basic data for your review.

Report and Revision Options

The blue area gives basic information about the type of report you generated and a link to re-open and revise the form creation criteria.

Saving Your Data

There are two options for retrieving the report from the system.

• Save PDF - Will save the report as a PDF for you to download or view in your browser.
• Email PDF - Allows you to send the report as a PDF to anyone you want directly from the system.

Back to top

A Note About Flattened Data

On some reports you may notice an asterisk (*) beside the number, such as in the screenshots below.

This just means that there was at least one target whose data has been "flattened" on the test or campaign. Some targets are counted in multiple tests/groups, but only one phishing email was actually sent to the target.

Back to top