QualitativeReview —
openai/gpt-5.6-sol
e95f58360c
The paper does not report its historical seed sequence; this reproduction used the predeclared seeds 42-51, so dispersion and exact means need not match the authors' unknown runs. · The bundled fixed graph dictionaries materialize 5,278/4,676/44,327 unique undirected Cora/Citeseer/Pubmed edges, while paper Table 1 prints 5,429/4,732/44,338; node counts and public split sizes match and no substitute data were used. · The fixed official citation.py evaluates in process and does not persist model weights, so per-run observations and complete logs are retained but no checkpoint artifact exists. · The historical hyperopt search was intentionally not rerun under the signed protocol; 60 iterations were verified from paper and fixed tuning.py, and released tuned files were used. · Execution used the available modern CPU environment (PyTorch 2.8.0) rather than the historical PyTorch 1.x-era environment; the fixed code ran without metric failures, with expected isolated-row and sparse-constructor warnings retained in logs. · Claim coverage is partial: only the Cora SGC value was tested; all listed literature and author-run baselines remain unverified. · The paper does not report its historical seed sequence; the reproduction predeclared seeds 42–51. · Execution used CPU and PyTorch 2.8.0 rather than the historical environment, although test accuracy is not intrinsically hardware-specific. · The bundled graph has 5,278 unique undirected Cora edges while Table 1 prints 5,429; node counts and public train/dev/test sizes match, and the difference may reflect edge-counting conventions, but exact dataset equivalence is not fully established. · The paper does not define the statistic represented by “±”; the reproduced population and sample standard deviations were retained rather than assumed equivalent. · The plan was revised after a one-epoch smoke probe but before the full campaign; no evidence indicates that the smoke result selected the seeds, 100-epoch arm, evaluator field, or aggregation. · Only 1 claim measurement(s) were executed; 15 remain unassessed. · This execution covers only part of the compound claim; unassessed measurements: m-t2-cora-gcn-lit, m-t2-cora-gat-lit, m-t2-cora-gln-lit, m-t2-cora-agnn-lit, m-t2-cora-lnet-lit, m-t2-cora-adalnet-lit, m-t2-cora-deepwalk-lit, m-t2-cora-dgi-lit, m-t2-cora-gcn-own, m-t2-cora-gat-own, m-t2-cora-fastgcn-own, m-t2-cora-gin-own, m-t2-cora-lnet-own, m-t2-cora-adalnet-own, m-t2-cora-dgi-own. · Only 1 claim measurement(s) were executed; 15 remain unassessed. The executed Cora experiment supports the covered SGC result: ten official-entrypoint runs at 100 epochs for seeds 42–51 produced 81.0–81.1% test accuracy, with an arithmetic mean of 81.05% versus the reported 81.0%. The evaluator computes correct/total on the public test indices, and the aggregation retains all ten runs. However, the compound claim also covers the other literature and author-run Table 2 baselines; 15 of 16 claim-side measurements were not executed. Therefore the full claim remains inconclusive despite support for the Cora SGC component.
