Aligned Multi Objective Optimization
Authors: Yonathan Efroni, Ben Kretzu, Daniel R. Jiang, Jalaj Bhandari, Zheqing Zhu, Karen Ullrich
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Lastly, we conclude by providing empirical evidence of the improved convergence properties of the new algorithms. We implemented CAMOO, PAMOO and compared them to a weighting mechanism that uses equal weights on the objectives (EW)2. We track the performance by measuring the mean-squared error between the networks: MSE = 1 |D| x D (hθ(x) hθ (x)) . Figure 2 shows the convergence plots of our experiments. |
| Researcher Affiliation | Collaboration | 1Meta AI 2Technion. Correspondence to: Yonathan Efroni <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 Weighted-GD, Algorithm 2 CAMOO, Algorithm 3 PAMOO |
| Open Source Code | Yes | We implemented CAMOO, PAMOO and compared them to a weighting mechanism that uses equal weights on the objectives (EW)2. https://github.com/facebookresearch/ Aligned Multi Objective Optimization. We share an IPython notebook in which we implement our algorithms and was used to generate all plots accompanies this submission. |
| Open Datasets | No | We draw data from a uniform distribution D = {xi}i where xi Uniform([ 1, 1]di). We generate target data from a randomly initialized network t(xi) = hθ (xi) + 10. We sample 200 points from an independent uniform distribution xi Uniform([ 1, 1]20). We generate target data from a randomly initialized network t(xi) = hθ (xi) + 10. |
| Dataset Splits | No | The paper describes generating synthetic data by sampling 200 points from a uniform distribution, but it does not specify any training, validation, or test dataset splits for these generated points. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. It describes network architectures and training processes but omits hardware specifications. |
| Software Dependencies | No | The paper mentions using SGD and ADAM as optimizers and states that an IPython notebook is shared. However, it does not provide specific version numbers for any key software components such as programming languages, deep learning frameworks (e.g., PyTorch, TensorFlow), or libraries. |
| Experiment Setup | Yes | For both problems, we set the learning rate of SGD to 0.0005 and ADAM to 0.005, we use the same learning rates for PAMOO. For CAMOOwe multiply the learning rate by the number of loss functions, 3. We set the number of samples for the Hutchinson method to be NHutch = 10. We set the number of iterations of the primal-dual algorithm to be 100 per-step. We set the learning rate to be 3e 3, and added a small regularization J x Jx J x Jx + τPAMOOI to avoid exploding weights, where τPAMOO = 1e 4. |