Deep Reinforcement Learning (DRL) has shown state-of-the-art performance in complex decision-making scenarios like computer chip design [1]. This paper introduces the Structural Optimization Gym (SOGYM), a novel Reinforcement Learning (RL) environment, specifically developed for training autonomous agents to design stiff structures through design space interaction. Expanding upon evidence indicating that interactive Topology Optimization (TO) environments improve the performance of novice human learner performance in compliance problems [2], our study compares this with the effectiveness of advanced DRL algorithms in a similar context. SOGYM leverages feature-mapping methods [3] to provide an efficient interface for artificial agents to modify the topology. Distinct from typical machine learning methods prone to overfitting to the training dataset [4], SOGYM continuously introduces diverse challenges, promoting versatile design strategies. Importantly, it integrates the physics of the problem into the agent's reward scheme, aligning with the actual TO objectives, and diverging from standard regression-based approaches. The training methodology involves a two-phase pre-training, including behavioral cloning and adversarial inverse reinforcement learning, followed by Proximal Policy Optimization (PPO) refinement [5]. Our initial findings indicate that while artificial agents in SOGYM exhibit the ability to learn structural load paths, their performance falls short of established density-based TO methods. Additionally, they display a lower sample efficiency compared to human learners. This early-stage research highlights SOGYM's potential as an open-source foundation for research in AI-assisted structural design, merging autonomous agents with engineering principles and setting the stage for developing more effective machine-learning methods in structural design.