Answer questions with data.
Problem: It isn’t possible to easily analyze the data from public facing systems.
Solution: Scrape data from the publicly available web service and perform analysis on it.
Example: Hamden Service Request Analysis
1. Scrape the data
Scrape data from a website or web service using the Python requests module.
Format the data in a format like CSV.
Consider making this raw data available to the public through an Open Data portal.
2. Develop questions
What questions can be answered by the data you’ve just scraped?
Which ones are most compelling or serve the greatest public interest?
Write these questions down.
3. Analyze data
There are many tools that can be used to answer your questions. Some are free (Google sheets) and some are paid (Excel & PowerBI).
If you are working with data without physical locations that do not need to be mapped, spreadsheet programs can provide great insights.
For example, in the service request data set scraped from SeeClickFix, you could subtract the date the issue was closed by the date the issue was opened to determine how many days it took to close each issue. From there you could calculate the average or median days for the Town to resolve reported issues.
4. Publish findings
The findings and answers to the questions you’ve developed should be published along with the raw data sets you used.
Consider sharing these insights with your local government and on social media.