Dear Marmam friends, For those involved in cetacean photo-ID, I write to invite you to join an automated image recognition algorithm development effort. In 2019, supported by Google’s Kaggle platform, we hosted an extremely successful algorithm development competition for humpback whale fluke-ID — the results: a fully automated deep convoluted neural network-based individual ID algorithm that in our tests compared to manual image matching, reduces image management time by at least 98%, and reduces error rates from approximately 6-9% to 1-3%. We learned a lot in the process, and image recognition has progressed such that, again with Google's support, we will host an image recognition algorithm development competition for lateral view cetacean photo-ID images — that is, dorsal fins and flanks from a lateral view perspective. Our goal is to produce one or several highly accurate and efficient lateral view individual ID algorithms, with open sourced results, and with successful algorithms accessibly deployed. Resulting algorithms may be species specific or may be able to manage multiple species, depending on results. We can’t guarantee success, but experience suggests that we will end up with very effective automated image recognition algorithms for any well represented species in the development dataset. If you are a catalog holder who could benefit from automated image recognition for cetacean photo-ID, please consider joining this collaboration. We will need images by mid-August at the latest. Here below are details of proposed collaborative image use for the algorithm development competition: Our only use of images will be for a Google sponsored Kaggle competition, similar to and building on our extremely successful effort in late 2018/early 2019 — https://www.kaggle.com/c/humpback-whale-identification <https://www.kaggle.com/c/humpback-whale-identification>. This competition fielded entries from 2129 teams, yielding the current best in class humpback fluke ID algorithm (we implemented the 3rd place winning algorithm), with an accuracy of 97-99% potential matches found in the first proposed result with good to high quality images (manuscript in prep). This algorithm is implemented into the Happywhale information architecture, enabling us to build a fluke ID dataset of, as of today 39,650 individuals, now accounting for an estimated 20% of the world's humpbacks, with no signs of loss of accuracy from too many individuals. This dataset is particularly rich on North Pacific humpbacks, where we believe we have identified > 50% of adult humpbacks alive from the year 2000 to today. Our goal is to produce one or several highly accurate and efficient lateral view individual ID algorithms, with winning results open sourced. To us, success will be a highly effective, efficient and accessible set of photo ID tools to further your work, to add value to mark recapture studies, and improve marine conservation science. We will aim to build an image set representing multiple species such that winning algorithms will effectively utilize shape, edge features, surface features, texture and patterns, thereby hopefully resulting in neural networks that can be trained for as-yet unrepresented cetacean species. Based on the success of the previous competition, I have high expectations for the results; the previous competition rapidly approached 100% accuracy early in the four month window, and in the intervening year and a half since that time, object recognition technology — and development accessibility — has made significant strides largely driven by investment in driverless car tech and similar automation efforts. To be clear, there’s no guarantee of success, either that resulting algorithms will be highly accurate or that they will be implementable. I believe however that it is reasonable to expect highly accurate results for any well represented species that is identifiable by the human eye. The team at Kaggle is excited for this, as they were very happy with the outcome from the previous competition, and we’ve learned a fair bit and talked a fair bit since then as to how to step onward together in this, so we can be assured of solid support on the host end. Image contributing collaborators will be included as co-authors in any resulting publications, with opportunities, if desired, to participate in the competition (there are roles for competition hosts to answer questions etc about the biological and environmental reality of photo ID, for example), first access to algorithm development and forthcoming system implementation, as well as any manuscript authorship, editing and review. The competition will begin in approximately October of this year, open for a period of 3-4 months. Competition images will be stripped of all metadata (we will do so), presented in the competition dataset divided between a set for algorithm training, with ID information (re-assigned to have no relation to IDs in existing datasets, whether public or private), and one set as test data, with no ID information. It should be recognized that while these images are stripped of all context, they will be made public for the sake of access by competitors. For a well written exploration of the competition process, from the point of view of the 10th place finisher in the previous effort, here’s a recently published read: https://towardsdatascience.com/a-gold-winning-solution-review-of-kaggle-hump... <https://towardsdatascience.com/a-gold-winning-solution-review-of-kaggle-humpback-whale-identification-challenge-53b0e3ba1e84>. Your use of images will not be limited in any way, apart from if you have an online catalog with the same images present, we should look at if this will potentially create a conflict that could allow ID process cheating among competitors. From you we would like to have as many images as possible, with a maximum of 20,000 ID’d images per species. For rare / difficult to photo-ID species there may be no minimum — this will be case specific — but for more commonly photographed species, a minimum contribution should be in the range of a few hundred individuals each photographed in different encounters. The ideal dataset includes a mix of cropped and uncropped images and a range of quality from very high to very poor. For simplicity, there should be just one individual in any image, cropped or uncropped. We want a natural distribution of images; some individuals should be represented by only one image (forcing competitors to accommodate the realistic designation of a ’new individual’ class), while some should be represented by many images (from separate encounters, to avoid any contextual matching, such as similar water texture / color). Because a lateral view inherently has the right-dorsal and left-dorsal set, two semi-independent classes per individual, we’d want a mix here as well. If you would like to join as a collaborator, we will need images by mid-August, sooner if possible so that the Kaggle team can begin preparing and developing the competition infrastructure (there’s some interesting questions on this end, such as dealing with some weighing of species by different data set sizes etc). Images can come to us as jpgs either (1) with the IDs indicated in a clear consistent format in the filename, (2) with IDs in a spreadsheet correlated by exact filenames, or (3) as a set of images with ID + date/location attributes that we’d then integrate into Happywhale (as a private dataset at least during the term of the competition) to format for the competition. We do not need annotation of R/L dorsal, etc. We will strip all metadata before adding images to the competition dataset. Of course I am available to discuss any questions, concerns or ideas around this. I am excited for your participation and hopeful for a very constructive outcome. With thanks for your interest, Ted — Ted Cheeseman ted@happywhale.com www.Happywhale.com https://www.facebook.com/happywhales/ ** know your whales :) **