Coherent Filtering:Detecting Coherent Motions from Crowd Clutters

1. Motivation

Coherent motion is a universal phenomenon in nature and widely exists in many phsical and biological systems. For example, the tornadoes, storms, and atmospheric circulation are all caused by the coherent movements of physical particles in the atmosphere. Meanwhile, the collective behaviors of organisms such as schooling fishes and pedestrian crowd have long captured the interests of social and natural scientists. Here are some examples of coherent motions:

Figure 1: Coherent motions widely exist in crowd.

The objectives of this project are

• revealing a prior called Coherent Neighbor Invariance underlying the coherent motions of particles;
• proposing an effective technique called Coherent Filtering for coherent motion detection in noisy time-series data.

Demo Video of Coherent Filtering,

Demo Video of Coherent Filtering: Detecting Coherent Motions. It could be downloaded from here

2. The Prior: Coherent Neighbor Invariance

Coherent Neighbor Invariance characterizes the local spatiotemporal relationships of individuals in coherent motion.

There are two key properties of Coherent Neighbor Invariance:

• Invariance of spatiotemporal relationships. The neighborship of individuals with coherent motions tends to remain invariant over time.
• Invariance of velocity correlations. The velocity correlations of neighboring individuals with coherent motions remain high when being average over time.

The following is the illustration of Coherent Neighbor Invariance.

Figure 2: Illustration of CNI in K nearest neighbor (K=7).The green dots are the invariant K nearestneighbors of the central black dot over time (here K = 7). The invariant neighbors have a higher probability to be the dots moving coherently with the central dot, since their local spatiotemporal relationships and velocity correlations with the central dot are inclined to remain invariant over time.

We further define two ratios to quantify Coherent Neighbor Invariance in K nearest neighbor through Random Dot Kinematogram, the existence of this prior is also verified.

3. The Technique: Coherent Filtering

Based on Coherent Neighbor Invariance, a clustering technique called Coherent Filtering is proposed for coherent motion deteciton from noisy time-series data. The core algorithm of Coherent Filtering is very simple, it consists three steps:

1. find K nearest neigbhor set at each time.
2. find the invariant neighbor and pairwise connection set.
3. threshold pairwise connection set by the average velocity correlations, then generate connected components as the coherent motion clusters.

Algorithm: CoheFilterDet for detecting coherent motion patterns.

Since coherent motion clusters will continue and new cluster of coherent motion will emerge over time. Based on the temporal overlaps of trajectories we develop another algorithm CoheFilterAssoci to associate and update the clusters of coherent motion over consecutive frames.

4. Experimental Results

We evaluate the robustness and effectiveness of Coherent Filtering on synthetic data, Hopkins motion segmentation database, and crowd videos.

On the synthetic data, we test the technique by detecting coherent motion patterns with different dynamics from high-density random walk noises.

Figure 3: Coherent motion in synthetic data. A) The shapes and the numbers of coherently moving dots(colors indicate different coherent motion patterns) and noisy dots(in blue). B) The dynamic traces of each coherent motion patterns. C) The detected coherent motion patterns by Coherent Filtering.

Then we evaluate the technique on the Hopkins155 Motion segmentation database.

Figure 4: Segmentaiton results on Hopkins155 database.

Experiments are conducted on 8 video clips with coherent motions emerging in different scales and distributions.

Figure 5: Coherent motion clusters detected in real scenes. Demo video could be downloaded from here. And the data of original videos and trajectories can be downloaded from here.

5. Reference

Citation of the paper is

Bolei Zhou, Xiaoou Tang, and Xiaogang Wang. "Coherent Filtering :Detecting Coherent Motions from Crowd Clutters." Proceedings of 12th European Conference on Computer Vision (ECCV 2012)