Chicago Wildlife Watch Talk

counting cars

  • ForestPreserve by ForestPreserve moderator

    This photo is good example of when to count a car. The car is clearly there in the landscape where cars don't normally go. You can see a comparison photo with no car in ACH000drje. It really looks like they're driving on the grass in this photo.

    Car that are driving on streets and roads don't count. Cars parked normally along those streets and roads don't count either.

    Cars parked in a park, as in this photo, do count. (So would car driving in park.)

    One exception to the roadway rule is alleys. The current rule is that cars in alleys do count.

    Unlike many cities in the eastern US, Chicago and its suburbs have lots of alleys, which are used for garbage collection and parking. At least a couple of cameras in the current set have alley views. One camera has a lovely view of some "Veolia" dumpsters. Another camera is located in a park that's bounded by alley, with row of garages on one side. For these alley cameras, counting cars and other vehicles makes sense.

    For most cameras, the "street view" is either incidental or the result of camera tampering. Don't count the cars in these photos.

    Posted

  • buffalogroveP by buffalogroveP

    What about the Metra trains? I am counting them as vehicle other, since I think they would have some impact on the animals. I have witnessed a coyote, in the daytime,waiting to cross the tracks when a train was going by up in my area ( northern suburbs of Chicago)

    Posted

  • DZM by DZM admin

    I think that the scientists are saying don't count vehicles when they are where vehicles are typically supposed (except for the alley exception)... so for a Metra train that is on its tracks, don't mark it.

    If the Metra comes off the tracks, yes, that should be marked! :p

    Posted

  • ForestPreserve by ForestPreserve moderator

    I've been counting trains too, but I'm certainly willing to stop. I've seen coyotes commuting along the Metra tracks in Evanston, too bad these traincams aren't picking up more critters.

    Posted

  • mason_UWI by mason_UWI scientist

    I agree with DZM here, we don't really need the train data. It's the train tracks that are the important part as they may function as a habitat corridor for some of our urban species, and we can collect that information from satellite images.

    Posted

  • escholzia by escholzia

    I think we've classified most of the train photos already, so a bit late to add this advice. And the brush is so thick in that spot, it's very unlikely we'd spot an animal.

    Posted

  • buffalogroveP by buffalogroveP

    I'd like to say THANK YOU for the directive that we don't need to note the cars and trains that are not on the actual path !! It makes things so much easier! and faster to just say nothing here!!

    Posted

  • mason_UWI by mason_UWI scientist

    No problem. I'm currently going through and classifying photos that users have not agreed 100% on (e.g. 4 people say its a coyote, 1 person says nothing) and will be writing up a post soon on it. Spoiler alert though, CARS ARE A BIG PROBLEM. I want to be able to more properly quantify it, but there may be a day in the future that the car ID no longer exists on CWW...

    Posted

  • escholzia by escholzia

    Not surprising - some of us (ahem) anticipated your decision to not have us count cars on roads by a few months.

    Posted

  • mason_UWI by mason_UWI scientist

    Once we get our 'test' data set analyzed and get a blog post up about it, expect to see some changes!

    Posted

  • DZM by DZM admin

    Hey, a lot of this involves learning as we go. 😃

    Posted

  • ForestPreserve by ForestPreserve moderator

    Big news on the counting cards front, as noticed by jpmapd. All of a sudden, there are no more options for "car", "vehicle, other", or "bike". So we can just ignore the trains, planes, and automobiles from now on, not to mention the garbage trucks and buses.

    We still have "mower" to keep you gearheads happy. And there's "horse", for those into green transport.

    Posted

  • ElisabethB by ElisabethB

    A huge thank you to the Powers that Be ! 😄

    Posted

  • mason_UWI by mason_UWI scientist

    The blog post is up. Check it out!

    Posted

  • buffalogroveP by buffalogroveP

    Very interesting blog post !!! thanks !

    Posted

  • DZM by DZM admin in response to mfidino@lpzoo.org's comment.

    Wow, that was a great read.

    At the end of the day, does it mean that the data is helpful? 😃

    Posted

  • mason_UWI by mason_UWI scientist in response to DZM's comment.

    ABSOLUTELY. Flagging a number of photos for expert ID is important, but we can still quantitatively deal with photos that may be false positives (saying a species is there when it is not) with more robust statistical methods. This data set is not only going to be incredibly useful from a quantitative side (e.g. dealing with potential bias that may be introduced through citizen science) there are some really cool ecological questions that we are asking with it. Some scientists don't like citizen science because they do not think the data is reliable, and while there may be a bit of noise that gets added, it can 'easily' be dealt with. Additionally, not only do we get to do a project that our staff could not do on our own, we get the opportunity to be part of a community that is interested in what we are interested in and get to share what we learn along the way!

    All in all, these data are amazing.

    Also, please note that on the home page there is now a 'blog' link that will direct you to the homepage for our blog posts. I will tell everyone in the talk section when a new one comes up, but make sure to check on it from time to time yourself!

    -Mason

    Posted

  • escholzia by escholzia

    If you can calculate agreement within 10% on a photo, each photo must be classified by ten people. That's a lot higher than most Zooinverse projects. Even considering that you remove "humans" after one classification, that's a load of classifications. Or are you simplifying your explanation?

    Posted

  • WillowSkye by WillowSkye

    Please explain the Stats some more (not my strong point, wish to understand the process used here). Are you saying that the regular Agreement Interval of 10 % is directly linked to the number of people required to classify an image? So if the Agreement Interval on the Graphs in the Blog was 5 %, only 5 people would be required to classify a photo? But it doesn't matter if 5/5 or 2/5 got it right (statistical modelling will deal with that and categorize accordingly in percentage bracket of accuracy) as long as 5 people classified the image? However, I'm assuming from the graph showing the number of photos tagged for each category, that "human" photos numbered approx. 1000 so far. Does this mean 597753 total images minus 1000 = 596753. 63 % completed equals plus/minus 376584 images completed. But if 10 people are required to classify an image, wouldn't it be 596753 x 10 = 5,967,530 classifications required? With ± 1,474,544 classifications already made, how is the 63 % calculated?

    Posted

  • DZM by DZM admin in response to WillowSkye's comment.

    This may not be a complete answer, but photos with "nothing here" are also retired quickly, and we have a lot of those...

    Posted

  • mason_UWI by mason_UWI scientist

    We have calculated 'agreement percentage' as such:

    1. Take all of the tags from each user and sum common tags together
    2. Divide by the number of users that tagged the photo

    So, for example, a photo of a raccoon was seen by 5 people, 4 of them tagged it as raccoon and 1 of them tagged it as nothing. The agreement percentage on this photo would be 80% (4/5).

    This process is not being used to determine how many times a photo needs to be seen by people, only if the photo needs to be verified by an expert once it has been retired. Even if you have more people view a photo, difficult to ID photos are still going to have less agreement because they are difficult. Additionally, easy photos don't need to be seen 10 times to determine what is in the image.

    As agreement percentage for a single photo increases (more people agree on what is in the photo) than the likelihood of the photo being correctly identified also increases. What this means to us is that photos need to be verified by an expert when people do not agree on what is in the photo. Instead of just randomly choosing some cut off point, I looked to quantify the relationship between agreement percentage and correct identification so that we can choose a cut-off point that we are comfortable with. In this case, having 90% certainty on what is present is a great starting point. In the future, when we use these data in modelling species distributions, we are going to go even further to quantify differences between citizen-science entered data and expert entered data. This will require some rather complex hierarchical modelling techniques, so I'm going to hold off on explaining 'the next step' right now. All of the modelling done for that blog post was done to illustrate how agreement percentage of a photo is related to correct identification on a photo, we have not thought about using it as a way to change the retirement rules for image classification.

    Here is the protocol for image retirement, which is why we are up to over 1 million classifications:

    • If 7 users have annotated the same animal, retire it.

    • If anyone has annotated a human, retire it.

    • If the first three users annotated the subject as blank, retire it.

    • If 5 users annotated the subject as blank, retire it.

    • If the classification count has reached 15, retire it.

    • Else, don't retire it.

    Currently, 376,584 photos have been retired (63% of 597,753). This has taken almost 1.5 million classifications. Thus, each photo is viewed an average of ~4 times (1,475,007 / 376,584). This mean that there are probably a decent number of 'human' photos and 'nothing here' photos that are bringing down the average number of times a photo is viewed. If the photos were primarily of wildlife, we would expect that average to be closer to 7 (the number of times a photo needs to be tagged with the same animal tag for retirement).

    Does that help with everyone's questions?

    Posted

  • WillowSkye by WillowSkye

    Thanks, this is an excellent explanation 😃 Wish you had been my Stats Lecturer at Varsity, I dropped it after 1 semester as I couldn't understand what he was saying, but with the above explanation I understood what you are saying. So maybe Stats is not so bad after all

    Posted

  • escholzia by escholzia

    Very complete, thanks 😃 I enjoy seeing what goes on under the hood of these projects.

    Posted