Efficient Occlusive Components Analysis
Authors: Marc Henniges, Richard E. Turner, Maneesh Sahani, Julian Eggert, Jörg Lücke
JMLR 2014 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To numerically investigate the algorithm for more realistic data, it was applied to data based on pictures of objects from the COIL-100 database (Nene et al., 1996).1 Images were scaled down to 20 20 pixels and were placed at random planar positions on a black background image of D = 35 35 = 1225 pixels. The objects were then colored with their mean color, weighted by pixel intensity (see Figure 6A). In 100 runs, we created N = 8000 data points by combining Hgen = 20 of these images according to the generative model with prior parameter π = 2 Hgen = 0.1, i.e., πHgen = 2 and data noise σ = 0.25 (see Figure 6B). For learning, the algorithm was applied with H = 30 mask and feature parameters Wh and Th, i.e., 50% more than we used for data generation. Figure 6C shows the resulting mask and feature parameters for an example run (where we display each pair of feature and mask combined into one image, compare Equation 3). We obtained all 20 underlying basis functions along with 10 noisy fields in 44% of the trials. |
| Researcher Affiliation | Collaboration | Marc Henniges EMAIL Frankfurt Institute for Advanced Studies Goethe-University Frankfurt, 60438 Frankfurt, Germany Richard E. Turner EMAIL Department of Engineering University of Cambridge, Cambridge, CB2 1PZ, UK Maneesh Sahani EMAIL Gatsby Computational Neuroscience Unit University College London, London, WC1N 3AR, UK Julian Eggert EMAIL Honda Research Institute Europe Gmb H 63073 Offenbach am Main, Germany J org L ucke EMAIL Cluster of Excellence Hearing4all and Faculty VI University of Oldenburg, 26115 Oldenburg, Germany |
| Pseudocode | No | The paper describes the M-step and E-step equations and their iteration process mathematically (e.g., Section 3.1 M-Step Equations, 3.2 E-Step Equations), but it does not present these steps in a structured pseudocode or algorithm block. |
| Open Source Code | No | The paper does not contain an explicit statement about the release of source code, nor does it provide a link to a code repository. |
| Open Datasets | Yes | To numerically investigate the algorithm for more realistic data, it was applied to data based on pictures of objects from the COIL-100 database (Nene et al., 1996). |
| Dataset Splits | No | N = 1000 images were generated for learning and Figure 4A shows a random selection of 10 noiseless and 10 noisy examples. The learning algorithm was applied to the colored bars test with H = 8 hidden units and D = 16 input units. In 100 runs, we created N = 8000 data points by combining Hgen = 20 of these images according to the generative model with prior parameter π = 2 Hgen = 0.1, i.e., πHgen = 2 and data noise σ = 0.25 (see Figure 6B). The data-set comprised 500 pictures of scenes consisting of up to five (toy) cars in front of a gray background. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, or memory) used for running the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers. |
| Experiment Setup | Yes | For all data points, a vector yd [0, 1]3 represented the RGB values of a pixel. In all trials of the experiments we initialized the parameters Whd and T c h by independently and uniformly drawing from the interval [0, 1]. The parameters for sparseness and standard deviation were initialized as πinit = 1 H and σinit = 5, respectively. Parameter optimization in multiple-cause models is usually a non-convex problem. For the OCA model, the strongly non-linear combination rule seems to result in even more pronounced local optima in parameter space than is the case for other models such as sparse coding. To efficiently avoid convergence to local optima, we (A) applied deterministic annealing (Ueda and Nakano, 1998; Sahani, 1999) and (B) added noise to model parameters after each EM iteration. Annealing was implemented by introducing the temperature T = 1 β. The inverse temperature β started near 0 and was gradually increased to 1 as iterations progressed. It modified the EM updates by substituting π πβ, (1 π) (1 π)β, and 1 σ2 β σ2 in all E-step equations. We also annealed the occlusion non-linearity by setting ρ = 1 1 β; however, once β became greater than 0.95 we set ρ = 21 and did not increase it further. We ran 100 iterations for each trial of learning. The inverse-temperature was set to β = 2 D for the first 15 iterations, then linearly increased to β = 1 over the next 15 iterations, and then kept constant until termination of the algorithm. Additive parameter noise was drawn randomly from a normal distribution with zero mean. Its standard deviation was initially set to 0.3 for the mask parameters and at 0.05 for the prior and noise parameters. The value was kept constant for the first 10 iterations and then linearly decreased to zero over the next 30 iterations. The degeneracy parameter α was initialized at 0.2 and increased to 0.6 from iteration 25 to 35. The amount of data points used for training was linearly reduced from N to Ncut between iteration 15 to 30. Approximation parameters were set to γ = 3 and H = 5 unless stated otherwise. |