Pope Tech: Scan Websites or Applications Behind Authentication

You may be able to use Pope Tech to scan websites or web applications that are behind authentication. This article is for websites or applications that are not built with Drupal or Google Sites. For Drupal sites, see Pope Tech: Scan Drupal Intranet Content. For Google Sites, see Pope Tech: Scan Google Sites Intranet Content. Knowledge of HTML and your website structure is required to complete the setup.

Table of Contents

• Ensure Functional Testing Account
• Provide Pope Tech Access to Your System
• Authentication Setup Tips

Notes:

• Only data classified as public per the University data security classification policy may be scanned with Pope Tech.  
• Support from OIT is best effort; we cannot guarantee that your website or application can be scanned by Pope Tech. 
  • As an alternative to Pope Tech, you can use a browser extension such as WAVE, axe DevTools, or others, to scan content one page at a time.  

Ensure Functional Testing Account

You will need a UMN functional account that has view access to your website or application for testing with Pope Tech. You will need to have access to the password to this account. 

1. If you already have a functional account that you can use for this purpose, skip to step 3.
2. Request a functional account to use for Pope Tech testing with your website(s).
   1. On the request form, select the following:
      1. Which OIM environment should the account be created in?
         1. PRODUCTION (Account will be created in OIM PROD environment)
      2. What access should be provisioned for this Functional Account?
         1. LDAP (Lightweight Directory Access Protocol)
         2. Optional: Require TwoFactor Authentication
            Note: If you require TwoFactor Authentication, you will not be able to use the scheduled scan functionality in Pope Tech and will need to run all of your scans manually. 
3. Ensure that your testing account has view access to your website/application before proceeding.

Provide Pope Tech Access to Your System

Ensure that Pope Tech will not be blocked by your web application's firewall. You may need to allowlist the user agent below:

• PopeTech-ScanBot/1.0 (+https://pope.tech)

If you need to allowlist Pope Tech's IP address rather than the user agent, contact [email protected] to ask for the IP addresses.

Authentication Setup Tips

We recommend that you begin by reviewing the vendor's Configure Website Authentication documentation and the UMN documentation for Google Sites or Drupal as a starting point. Then see the additional tips below: 

• In all likelihood, you will need to check the box to Use Advanced Authentication? within the Authentication Settings. With this method, you provide step-by-step instructions for the tool to login to your site
• When specifying element selectors:
  1. Use # to indicate an id
  • Ex. #username can be used to indicate the INTERNET ID input field on login.umn.edu

2. Use . (a period) to indicate a class

• Ex .idp3_form-submit can be used to indicate the Sign In button on login.umn.edu

• You may need to add "Wait" actions at one or more points along the way to account for page load and processing time. 
  1. You'll be able to determine where these are needed based on where the authentication testing gets stuck based on the error screenshots. 
     
     Note: If you are using a functional account protected by TwoFactor Authentication, add a wait to account for a human to complete this step during the scan - i.e. a 30 second wait to allow sufficient time for receiving and approving a DUO push. 
• Check the Base URL field after testing authentication
  1. Pope Tech appends a path in your base URL field when it encounters a redirect during the sign-in process. If this path is not actually present on every page on your website that you want the tool to scan, follow these steps:
  • Remove the path from the Base URL field
  • Check the Ignore redirect? checkbox underneath the field
  • Save your changes before running a scan
• If your website or application is behind Microsoft's Entra SSO rather than Shibboleth, see below for the recommended advanced authentication steps:

1. Step 1

• Step Action: Type into input
• Element Selector: #i0116
• Input text: functional account UMN email address
  • i.e. [email protected]

2. Step 2

• Step Action: Click
• Element Selector: #idSIButton9

3. Step 3

• Step Action: Type into input
• Element Selector: #i0118
• Check Is this field a password? checkbox
• Input functional account password in Password field

4. Step 4

• Step Action: Wait for time
• Wait time: 3

5. Step 5

• Step Action: Click
• Element Selector: #idSIButton9

6. Step 6

• Step Action: Wait for time
• Wait time: 10

Add Paths to Scan

Pope Tech cannot crawl content behind authentication. You can manually add paths to scan via the Pope Tech user interface, or you can bulk add paths via a CSV upload. See the vendor's Add Pages article for instructions.