Main error:
PicklingError:
Can't pickle <class 'neurolib.utils.parameterSpace.ParameterSpace'>:
it's not the same object as neurolib.utils.parameterSpace.ParameterSpace
Full error:
Python executable: /Users/pbj/opt/anaconda3/envs/neurolib-final/bin/python
NumPy version: 1.23.5
PyTables version: 3.8.0
PyTables linked to HDF5: 1.12.2
MainProcess pypet.storageservice.HDF5StorageService INFO I will use the hdf5 file `./data/hdf/wcisp-gridsearch.hdf`.
MainProcess pypet.environment.Environment INFO Environment initialized.
MainProcess root INFO Number of parameter configurations: 3375
MainProcess root INFO BoxSearch: Environment initialized.
MainProcess pypet.environment.Environment INFO I am preparing the Trajectory for the experiment and initialise the store.
MainProcess pypet.environment.Environment INFO Initialising the storage for the trajectory.
MainProcess pypet.storageservice.HDF5StorageService INFO Initialising storage or updating meta data of Trajectory `results-2025-04-10-17H-16M-02S`.
MainProcess pypet.storageservice.HDF5StorageService INFO Finished init or meta data update for `results-2025-04-10-17H-16M-02S`.
MainProcess pypet.environment.Environment INFO
************************************************************
STARTING runs of trajectory
`results-2025-04-10-17H-16M-02S`.
************************************************************
MainProcess pypet.storageservice.HDF5StorageService INFO Initialising storage or updating meta data of Trajectory `results-2025-04-10-17H-16M-02S`.
MainProcess pypet.storageservice.HDF5StorageService INFO Finished init or meta data update for `results-2025-04-10-17H-16M-02S`.
MainProcess pypet.environment.Environment INFO Starting multiprocessing with at most 8 processes running at the same time.
MainProcess pypet INFO PROGRESS: Finished 0/3375 runs [ ] 0.0%
Traceback (most recent call last):
File "/Users/pbj/Documents/GitHub/srg-wb-tms-modeling/analysis/wcisp_gridsearch.py", line 91, in <module>
search.run(
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/site-packages/neurolib/optimize/exploration/exploration.py", line 313, in run
self.env.run(self.evalFunction)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/site-packages/pypet/environment.py", line 1806, in run
return self._execute_runs(pipeline)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/site-packages/pypet/environment.py", line 2330, in _execute_runs
self._inner_run_loop(results)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/site-packages/pypet/environment.py", line 2463, in _inner_run_loop
expanded_by_postproc = self._execute_multiprocessing(start_run_idx, results)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/site-packages/pypet/environment.py", line 2777, in _execute_multiprocessing
proc.start()
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Users/pbj/opt/anaconda3/envs/neurolib-final/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'neurolib.utils.parameterSpace.ParameterSpace'>: it's not the same object as neurolib.utils.parameterSpace.ParameterSpace
Here's the full code:
# wcisp_gridsearch_final.py
# --- Core Imports ---
import sys
import os
import dill
os.environ["NO_PROXY"] = "localhost,127.0.0.1" # Prevent network-related hangs
os.environ["OBJC_DISABLE_INITIALIZE_FORK_SAFETY"] = "YES" # macOS specific
import numpy as np
from multiprocessing import set_start_method
import gc
import importlib
# --- Configure multiprocessing first ---
set_start_method("spawn", force=True)
# --- Add local module path ---
sys.path.append('/Users/pbj/Documents/Github/srg-wb-tms-modeling/wbm')
# --- Import your custom model ---
from wcisp import WCISP
# --- Neurolib Setup ---
from importlib import reload
from neurolib.optimize.exploration.exploration import BoxSearch
import neurolib.utils.parameterSpace as ps # Import the MODULE, not just the class
from neurolib.utils.parameterSpace import ParameterSpace
importlib.reload(ps)
ParameterSpace = ps.ParameterSpace
# --- Environment Configuration ---
os.environ["OMP_NUM_THREADS"] = "1" # Disable OpenMP threading
os.environ["HDF5_USE_FILE_LOCKING"] = "FALSE" # Prevent HDF5 locking issues
# --- Debug Information ---
print("\n=== Environment Verification ===")
print(f"Python executable: {sys.executable}")
print(f"NumPy version: {np.__version__}")
import pickle
pickle.Pickler = dill.Pickler
pickle.Unpickler = dill.Unpickler
try:
import tables
print(f"PyTables version: {tables.__version__}")
print(f"PyTables linked to HDF5: {tables.get_hdf5_version()}")
except ImportError as e:
print(f"PyTables import failed: {e}")
raise
# --- Main Execution Block ---
if __name__ == "__main__":
# --- Load connectivity data ---
Cmat = np.loadtxt('Abeysuriya_cmat.csv', delimiter=',')
Dmat = np.loadtxt('Abeysuriya_dmat.csv', delimiter=',') * 100 # Convert to cm
# --- Test model ---
test_model = WCISP(
Cmat=Cmat,
Dmat=Dmat * 0.1, # Convert cm to mm
duration=30,
exc_ext_baseline=0.31,
K_gl=0.22,
c_ie_0=-3.8
)
test_model.run()
# --- Parameter Space ---
params = ParameterSpace({
"K_gl": np.linspace(0.1, 1.0, 5),
"c_ee": np.linspace(2.5, 4.5, 3),
"c_ei": np.linspace(2.5, 4.5, 3),
"rho": [0.1, 0.15, 0.2],
"exc_ext_baseline": np.linspace(0.1, 0.5, 5),
"c_ie_0": np.linspace(-4.5, -1.5, 5),
}, kind="grid")
gc.disable()
# --- Grid Search Setup ---
search = BoxSearch(
model=WCISP(Cmat=Cmat, Dmat=Dmat * 0.1, duration=30),
parameterSpace=params,
filename="wcisp-gridsearch.hdf"
)
# --- Safe Execution ---
search.run(
multiprocessing=False, # Disable until environment is stable
chunkwise=True,
bold=True,
backend="sequential"
# Reduce verbosity
)
print("\nGrid search completed successfully!")
I've tried a few things: Cretaing new environments, and installing compatible versions of packages as well environment hardening
export PYTHONHASHSEED=0
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
Not sure what is wrong, would love some help! Have been stuck on this for quite some time now. I have also tried troubleshooting by including a few checks in the code itself but so far nothing has worked.
Pythonor differentneurolibthen this can make problem.