Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Logical Credal Networks
Authors: Radu Marinescu, Haifeng Qian, Alexander Gray, Debarun Bhattacharjya, Francisco Barahona, Tian Gao, Ryan Riegel, Pravinda Sahu
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our proposed approach on random LCNs as well as benchmark problems derived from Mastermind puzzles with uncertainty and a realistic credit card fraud detection application. All our experiments were run on a 2.6GHz CPU with 32GB of RAM. |
| Researcher Affiliation | Industry | Radu Marinescu IBM Research EMAIL Haifeng Qian AWS AI Labs EMAIL Alexander Gray IBM Research EMAIL Debarun Bhattacharjya IBM Research EMAIL Francisco Barahona IBM Research EMAIL Tian Gao IBM Research EMAIL Ryan Riegel IBM Research EMAIL Pravinda Sahu IBM Consulting EMAIL |
| Pseudocode | No | The paper describes algorithms but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source code is included in the supplementary material |
| Open Datasets | Yes | We consider a realistic credit card fraud detection task based on the UCSD-FICO Data Mining Contest dataset [13] |
| Dataset Splits | No | The paper describes training and test splits for the credit card fraud detection dataset, but it does not explicitly provide details about a validation dataset split for any of the experiments. |
| Hardware Specification | Yes | All our experiments were run on a 2.6GHz CPU with 32GB of RAM. |
| Software Dependencies | No | The paper mentions using 'ipopt [31]' as a non-linear solver, but it does not specify a version number for this or any other software dependency. |
| Experiment Setup | Yes | For our purpose, we generate random LCNs with n propositional variables {x1, . . . , xn} and n + 3 sentences of the form l P(xi) u and l P(xi|xj) u, where l, u [0, 1] and u l 0.3. For each problem size n, we generate 10 random instances and for each instance we select 5 different pairs of variables (xi = xj) to formulate 4 queries per pair. (from Section 4.1) ... The LCN based formulation assumes that sentences (19) (21) are annotated with τ = False so that they do not imply dependency among the li variables. (from Section 4.2) ... For each formula and each of the 10 test sets, we measured the conditional probability that the consequent is true given that the antecedent is true. We took the min and max over the test sets and obtain the following probability intervals for the three rules: [0.65, 0.74] for (22), [0.31, 0.66] for (23) and [0.44, 0.72] for (24), respectively. (from Section 4.3) |