Home | GitHub | Meetup | Blog | Our Data | Slack

YOUR BRIGADE NEEDS YOU: Help us clean up crime data recently obtained from SCMPD


(Carl V. Lewis) #1

Georgia’s latest Open Records Laws available here:

Note in section A, article 2, it states:

‘Public record’ means all documents, papers, letters, maps, books, tapes, photographs, computer based or generated information, data, data fields, or similar material prepared and maintained or received by an agency or by a private person or entity in the performance of a service or function for or on behalf of an agency or when such documents have been transferred to a private person or entity by an agency for storage or future governmental use.

While I’m no legal scholar, I take that to mean that legally SCMPD cannot refuse to provide the specific, detailed data they are providing to LexisNexis for [communitycrimemapping.com](http://communitycrimemapping.com –– otherwise, they’d be in clear violation of Ga.'s Open Records laws.

:rescue_worker_helmet: YOUR BRIGADE NEEDS YOU: Help us clean recently obtained crime data spreadsheets from SCMPD
(Roy) #2

I know that one of the things discussed was attempting to meet with those responsible for aggregating this information. It would be good to map out the process so that we can align our retrieval efforts to the right step so that it is not a time burden on the PD to comply with a FOIA.

My person experience tells me that if we can make it easy for them to provide the info, the more likely they will go with it. H%ck, if you get PD leadership buy-in, giving that info to Open Savannah could become a part of the process.

(Carl V. Lewis) #3

Hi there, fellow civic web-makers, urban movers-and-shakers and masterpiece creators––

Good news! After having spent a great deal of time and effort trying to obtain crime data in a tabular format from Savannah-Chatham Metropolitan Police Department, I’m happy to let you all know we received the requested data yesterday via email. The only downsides?

  • The 13 different provided Excel files are far from clean enough to perform much useful analysis easily (there are erroneous images and art splashed around, and far too much summary formatting). We’re going to need a team of folks to volunteer to get the spreadsheets cleaned up and exported as .csv files.

  • The files do not contain the actual detailed crime summary reports with address approximations (i.e. the 300 block of Bull St.), just the weekly reports. While we can make another request for the full reports, this is a start… for now. One thing we remain resolute on: If that data can be provided to the less-than-transparent vendor ‘Lexis-Nexis: Community Crime Maps’ (formerly known as RAIDs)

Call to Action:

###Who wants to join me in maniupulating/refining this data into more structured schema that we can visualize easily for both analysis purposes and sharing purposes? If you want to be a data hero for a topic in high-demand from residents, join me in helping clean these Excel files up!

(Roy) #4

I will help with this, @carlvlewis

Much of it is processed from other fields out on other worksheets. Some is data that has been added together and some are averages(something that does not really need captured because we can do the math ourselves.)

Is the aim to save this summary data or will you want to be working with data from different beats? If that is so, I can figure out the document’s schema so we are getting data that can be processed ourselves.

(Roy) #5

Done with the Total Jurisdiction worksheets as they stand. Total Jurisdiction CSV

(Nick Helmholdt) #6

FYI we have an Open Data link to the map of police beats for SCMPD.

(Kevinsagis) #7

Nick - thanks for the plug! :smiley: Also they have a crime map here that data could be grabbed from - https://spotcrime.com/ga/savannah

(Carl V. Lewis) #8

@kevinsagis See the other thread about the issues with their crime mapping vendor…

(Carl V. Lewis) #9

Sweeet, @roy.e.mosby! Want to upload it to data.opensavannah.org?

(Roy) #10

I’ve updated the Police data summary to include the one detailed report.

(Nick Helmholdt) #11

Thanks Kevin, In our long-ish conversation about crime data that website never came up. Is this something that SCMPD has set up? It looks like it might be scraping data from the http://communitycrimemap.com/ website.

(Carl V. Lewis) #12

@NHelmholdt See my previous post which includes an analysis of Georgia’s Public Records Laws as well as the annotations and highlighted text I marked up on CommunityCrimeMapping Terms of Use page . (you’ll need to select the annotation pullout menu from the right of the page to see my comments).

Basically, LN explicitly obfuscates access to the data, and bans anyone from scraping it -– depite the fact it is public data. Additionally, LN has been reported to have agreements informally with the departments they work with that only LN will get the data. In my personal experience, when I requested the same data that LN receives from SCMPD, I was told simply to go to LN (it was then called RAIDs).

This piece from the Center for Investigative Reporting has some pretty damning indictments of LN’s crime map agreements.

(Carl V. Lewis) #13

@kevinsagis @NHelmholdt Yeah, what I was getting at in another post here is that we have everything we need to be able to calculate crime rates by neighborhood and display that in a choropleth map. SCMPD doesn’t calculate crime rates because that would require they know the population of their precinct boundaries –– so they only provide a tally of the numbers of crimes for each neighborhood.

Our task will be to match up Census Data tracts with the arbitrary neighborhood polygons of SCMPD to determine the population of each neighborhood.

Then we can calculate it fairly simply:


Here are the neighborhood boundaries in the three primary geospatial filetypes:

Here’s a pretty interactive map with popovers and all that jazz:

Once we have the rates, we can accurately map the levels of crime in each precinct with a choropleth (color-coded) scale. This has never been done before.