Comparing Patterns

Introduction

A key aspect of MantaMatcher for researchers is being able to compare the ventral spot patterns of mantas with each other, to determine if they are already in the database. Mantas have very diverse ventral spot patterns, making visual matching the most commonplace method of pattern matching, and what most researchers are likely to already be doing before submitting data to the website.

Another way to do pattern comparison is using the MantaMatcher Algorithm (MMA), which has been integrated into the website to assist in finding possible matches. To use MMA you'll need to create a special “candidate region” (CR) image of a manta's ventral spot pattern, which is then used by the algorithm. The requirements for this image are a bit stricter than for a normal identification image, and are outlined below.

Note: If you are a Regional Manager and you've not used the algorithm before, please read through this guide carefully to fully understand how to use it.

Note: You will see CR images also being called “feature region” images on the website. These terms are roughly interchangeable, where the “candidate region” refers to the cropped/aligned image uploaded by a user, and the “feature region” being the same image after pre-processing by the algorithm ready for use.

Using the MantaMatcher Algorithm (MMA)

Preparing images for the algorithm

The algorithm uses CR images to make comparisons between mantas. CR images are simply carefully cropped/aligned versions of good identification (ID) images, giving the algorithm the best chance to identify the key spot pattern features, so it usually makes sense to use the best ID image available for an encounter.

When deciding whether to include a CR image for an encounter, bear in mind:

  • Each CR image is associated with an existing image (where the spot pattern is visible).
  • Each encounter can have multiple CR images (but only one per normal image).
  • An encounter with no CR images cannot be used with the algorithm.
  • It's likely better to have no CR image for an encounter than only a bad one (and therefore should likely use the Visual Matcher tool instead).

A good CR image is as follows:

  • Has little back-scatter (reflective particles in water).1
  • Is taken close enough to avoid quality issues (e.g. plankton/fish obscuring view).1
  • Is taken perpendicular to the manta's ventral pattern (i.e. not at an angle)2
  • Uses a flash/strobe to better light the manta (as necessary).
  • Has been rotated & cropped to include only core spot pattern area in a vertical orientation.1
  • Does not include any/much extra irrelevant image data (e.g. wings, cephalic fins, water, background, etc.)1

1 Photos with irrelevant data can mismatch based on incorrectly interpreted artifacts.
2 Photos taken at an angle distort the pattern, making matching more unlikely.

The simplest way to create a CR image is directly from an existing ID image. Nearby your selected ID images there should be a link to create an associated CR image (or if one already exists, a thumbnail representing the existing CR image). By clicking this link you can use a tool to make a selection from the existing ID image. Alternatively the CR image for an associated ID image may be prepared offline, then uploaded on the encounter page in the Matching Algorithm section (near the bottom of the encounter page).

Once a CR image has been created/uploaded it may be seen in the Matching Algorithm section of the encounter page. What is shown is actually a “feature region” image, which gives an indication of how the algorithm has interpreted the features, using arrow vectors to represent the features it considers important. You are strongly recommended to examine this feature image each time you create a CR image to make sure it appropriately represents the manta features.

Candidate Region image creation: step-by-step

  1. Select ID image to use for conversion to CR image.
  2. Click link underneath ID image thumbnail (on upper right-hand side of encounter page) to launch the CR selector tool.
  3. Rotate/resize selection box to select suitable region of ID image for use as CR image.
  4. Accept the selection, and check no errors occurred during MMA processing of CR image.
  5. Return to encounter page, then validate resulting feature image for suitability.

Is my Candidate Region image good enough?

Deciding whether a CR image is good enough for use with the algorithm is a subjective choice, but care should be taken. Feature arrows on the displayed feature image should be most prominent (i.e. longest/largest arrows) originating from the key identifying features of the manta (e.g. a dark ventral spot). Larger arrows represent greater significance during algorithm matching, so if a prominent arrow originates from a non-identifying feature (e.g. a remora or shadow), this could very likely reduce match reliability.

Using the MantaMatcher Algorithm

Example **Matching Algorithm** section of an encounter page

The algorithm is available for use by researchers to help identify matches to existing encounters in the database. If you have permission to use the algorithm you will see the various options available in the Matching Algorithm section of each encounter page. Here you can:

  • Upload a new CR image (although usually done as detailed above).
  • Scan the database for possible matches with other MMA images.
  • View the results page of a previous scan (if one exists).
  • Rescan the database for possible matches with other MMA images.
  • Remove an existing CR image.

Each “feature image” displayed in this section shows a processed version of the image with a number of colored arrows. These arrows are a visual interpretation of how the algorithm has analyzed the image, and should give a good indication whether your image is too low in quality, or has too much irrelevant detail outside the pattern. If a “feature image” doesn't look like it represents the markings of the manta, it's best to remove it. For encounters with no CR images, they can be matched using the Visual Matcher tool instead.

For best results using the algorithm, you should ensure the encounter has valid entries for species (M.birostris or M.alfredi), pigmentation (patterning code), and sex (male/female/unknown). This can help to significantly narrow the list of potential matches.

There are two options for using the algorithm:

  • Regional - checks against all (algorithm-compatible) encounters with the same *Location ID* (good for day-to-day maintenance of a regional database).
  • Global - checks against all (algorithm-compatible) encounters in the database (good for occasional use to find potential matches).

Unsurprisingly the regional scan is faster than the global, as it doesn't have to compare against as many other encounters. Both types of scan pre-filter based on species/pigmentation/sex as explained above.

The results page shows a brief summary of the algorithm's analysis at the top. Below is a table of all the matches found, with the reference manta on the right (i.e. one being scanned for matches), and the potential matches shown on the left. The results are listed in descending order of their relevance. While the algorithm's “most likely” matches are at the top of the list, the inexact nature of the matching process means you can look down the list for other possible matches (particularly in cases where spot patterns are complex and/or indistinct). Each displayed image links to the encounter page for that image, allowing you to find more information.
Note: other images for the same encounter will not appear in the results.

Example MantaMatcher algorithm results page

Examples images: what's usable, and what's not

This section shows a few examples of original images and the feature images created from them, to give an idea of whether or not the feature images should be kept or discarded. While the choice is subjective, it benefits all researchers using MantaMatcher to make considered choices for the feature images.

Original Image Resulting Feature Image Description
Example 1: original image 1 Example 1: feature image 1 Features from both images seem to be represented reasonably well, despite the remora. Because it's a complicated spot pattern, and the spots are distinct, the algorithm has a lot of information to work with. A few spots are not represented, but on the whole these work well. Both feature images were kept.
Example 1: original image 2 Example 1: feature image 2
Example 2: original image 1 Example 2: feature image 1 Despite the grainy and slightly lower quality original, the feature image seems to adequately represent the manta's simple spot pattern, so it was kept.
Example 3: original image 1 Example 3: feature image 1 Original image is from a very oblique angle, but despite this the spot pattern clear for a visual match. However, for the algorithm, while the feature image seems to represent the spots ok, it also picks up a lots of other non-pattern related features, and was therefore discarded.
Example 4: original image 1 Example 4: feature image 1 Despite being a scan from the original slide film, and being at an angle, the spot pattern is clear to the eye. However the feature image shows one of the main spot markings not being recognized, so it was rejected.
Example 5: original image 1 Example 5: feature image 1 Another case where both feature images fail to represent several main spots in the pattern, so both were rejected.

This case highlights the benefit of both taking a photo perpendicular to the manta, and using a camera flash to improve the lighting, both of which could have helped here.
Example 5: original image 2 Example 5: feature image 2
Example 6: original image 1 Example 6: feature image 1 Clear original photos, although the first feature image represents the single main spot in the pattern poorly. The algorithm has picked up on some of the back-scatter in the foreground, shown by the small arrows at the bottom of each feature image, but because the arrows are short it implies it gives them little “weight” to the overall matching process compared to the longer/larger arrows. Together the three feature images seem to adequately represent the spot pattern, so they were all kept.
Example 6: original image 2 Example 6: feature image 2
Example 6: original image 3 Example 6: feature image 3

Using the Visual Matcher

The alternative to using the MantaMatcher Algorithm (MMA) is to use the Visual Matcher (VM) tool. This is an online tool to assist with making side-by-side image comparisons using the images uploaded for encounters.

Why would I use the VM instead of MMA?

Manta encounters are comparatively unpredictable events, and as frequent manta sighters know, it's often difficult to come back with ideal identification photos. If the ID photos are good enough to use to create good quality CR images for the algorithm, then using the algorithm is the preferred option, but as a fallback scenario the VM allows comparison in the more traditional way.

How do I use the VM?

You will need a stable and reasonably fast internet connection to use the VM, as it needs to download many encounter images for display.

  1. Load the encounter page you wish to try to match.
  2. Click the Visual Matcher link near the top of the encounter page.
  3. The VM will load, displaying two panels: Encounter Target Image, and Candidate Encounters.
  4. Use the drop-down boxes to narrow down the list of Candidate Encounters for comparison.
  5. Use the mouse (left click) or keyboard (Z) to perform zoomed comparison of images.
  6. If the candidate matches the target, click the appropriate button to confirm the match.

Bear in mind the VM is simply a tool to assist the process, and you still need to think about the images used as candidates for comparison. For example, you will very likely need to perform several rounds of comparison to ensure you've covered all the possible match options:

  • Target encounter: M. alfredi, normal pigmentation, male
  • Candidate filters (1st round): normal pigmentation, male
  • Candidate filters (2nd round): normal pigmentation, unknown sex
  • Candidate filters (3rd round): unknown pigmentation, male
  • Candidate filters (4th round): unknown pigmentation, unknown sex