This is a general introduction to Imatest test plans, which are being added starting in 2021 to provide customers with guidance on how to meet their image quality objectives.
We will be providing several boilerplate (standard) test plans for specific applications, such as testing zoom lens quality. Most of these plans will need to be customized, and we will provide guidance on the website and consultation on customization.
Step 1: What do you need to test (for your specific application)?
Each test plan should be tailored for a specific application. For example, for photogrammetry, (low) distortion is extremely critical. It may be relatively unimportant (unless it's excessive) for landscape photography.
Here are some of the questions that need to be answered.
- Will you be reading images from files, or is direct image acquisition available?
- What physical Field of View (FoV) is needed for the testing. Angular FoV may also be important.
- What is the distance (or distances) from the camera to the chart?
- What is the pixel size of the sensor?
- What is the physical size (in mm) of the sensor?
- Is the lens fixed focus, manual focus, and/or autofocus?
- Is it a zoom lens?
- Is the lens fixed or variable aperture (F-number)?
- What image quality factors (Key Performance Indicators) are most important for your application?
-
- Do you need to map sharpness over the image? If so, SFRplus, eSFR ISO, or Checkerboard (compared here) are recommended.
- Is distortion important? Checkerboard gives the most accurate results, but SFRplus is good for many purposes. eSFR ISO is very limited.
- Is color accuracy and tonal response important? You may need a separate chart with precise colors and a grayscale such as the inexpensive and widely-used X-Rite Colorchecker.
- Is noise and SNR (Signal-to-Noise Ratio) important? (It almost always is.) eSFR ISO and most separate charts that contain grayscales should work.
- Do you need to know Dynamic Range? If so, what DR is required? What is the DR in the actual use case? Note that HDR (High Dynamic Range) image sensors are often specified at much higher DR than real-world cameras (with lenses and flare light) can actually achieve. This has led to frequent misunderstandings.
- Can you read raw images?
- Do you need flat-field (uniformity) testing? (Can be useful for system tuning— ensuring that color and luminance response are uniform.)
- Do you need to measure motion blur?
- Is your camera tone-mapped? (This muddies tonal response measurements, but Contrast Resolution gives useful results.)
- Does your camera have bilateral (nonuniform) filtering that sharpens edges but blurs smooth areas? This can compromise texture performance. A texture chart such as Spilled Coins or Log F-Contrast may be needed.
- Will your system be used for a specific task, such as object recognition? If so, the object of interest (or a replica) may need to be included in the image (either next to a chart or covering an unused portion of the chart).
- Based on the answers to the above questions, one or (usually) more charts (chart type, size, and media) may be selected. Some guidance can be found in Using Imatest - Getting started. If you write our charts department for advice, be sure to let them know your working distance, Field of View, and sensor pixel size.
- What lighting and light levels are needed? Does the camera have built-in illumination (which may need to be turned off for some tests)? Lighting is characterized by color temperature and intensity at the working distance (and occasionally spectrum).
- What associated equipment (for example, test stand) is needed?
Step 2: Assemble and set up the camera and test equipment.
This will be covered in the individual test plans.
Step 3: Acquire the test images
This will be covered in the individual test plans.
Step 4: Analyze the images and set up Pass/Fail criteria
This step will be covered in detail in Test Plan Reference pages for several test charts in several .
Test plan reference: Slanted-edge modules. This discusses running SFRplus, eSFR ISO, or Checkerboard (compared here) using eSFR ISO as an example.