CUDA error: an illegal memory access was encountered

#1
by fspecii - opened

Seems like we're getting CUDA error: an illegal memory access was encountered
<|im_start|>user

Caption

dark trap, 808u

Lyric

<|im_end|>
<|im_start|>assistant

Generating: 0%| | 0/1 [00:00<?, ?steps/s]
Generating: 0%| | 0/1 [00:00<?, ?steps/s]
2026-01-30 22:46:11.497 | INFO | acestep.inference:generate_music:377 - [generate_music] LLM usage decision: thinking=True, use_cot_caption=True, use_cot_language=True, use_cot_metas=True, need_lm_for_cot=True, llm_initialized=True, use_lm=True
2026-01-30 22:46:11.497 | INFO | acestep.inference:generate_music:435 - LM chunk 1/1 (infer_type=llm_dit) (size: 2, seeds: [3550754306, 2083999114])
2026-01-30 22:46:11.497 | INFO | acestep.llm_inference:generate_with_stop_condition:897 - Batch Phase 1: Generating CoT metadata (once for all items)...
2026-01-30 22:46:11.500 | INFO | acestep.llm_inference:generate_with_stop_condition:905 - generate_with_stop_condition: formatted_prompt=<|im_start|>system

Instruction

Generate audio semantic tokens based on the given conditions:

<|im_end|>
<|im_start|>user

Caption

R&B with smooth vocal-style melodies, warm synth chords, grooving basslines, crisp drums, and soulful harmonies blending modern rhythm with classic soul emotion.

Lyric

[Intro R&B with lush synth pads, mellow chords, and subtle rhythmic groove]

[Instrumental R&B with flowing melodies, warm harmonies, and smooth groove-driven rhythms]
<|im_end|>
<|im_start|>assistant

Generating: 0%| | 0/1 [00:00<?, ?steps/s]
Generating: 0%| | 0/1 [00:00<?, ?steps/s]
2026-01-30 22:46:29.095 | INFO | acestep.llm_inference:create_sample_from_query:1561 - Creating sample from query: Rap... (instrumental=[], vocal_language=unknown)
2026-01-30 22:46:29.097 | DEBUG | acestep.llm_inference:create_sample_from_query:1568 - Formatted prompt for inspiration: <|im_start|>system

Instruction

Expand the user's input into a more detailed and specific musical description:

<|im_end|>
<|im_start|>user
Rap

instrumental: false<|im_end|>
<|im_start|>assistant

2026-01-30 22:46:29.097 | DEBUG | acestep.constrained_logits_processor:set_target_duration:1226 - Target duration cleared, no duration constraint
2026-01-30 22:46:29.097 | DEBUG | acestep.constrained_logits_processor:set_user_metadata:424 - No user-provided metadata, all fields will be generated
Generating: 0%| | 0/1 [00:00<?, ?steps/s]
Generating: 0%| | 0/1 [00:00<?, ?steps/s]
Traceback (most recent call last):
File "/home/user/.local/lib/python3.11/site-packages/gradio/queueing.py", line 766, in process_events
response = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/route_utils.py", line 355, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/blocks.py", line 2147, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/blocks.py", line 1641, in call_function
prediction = await utils.async_iteration(iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/utils.py", line 859, in async_iteration
return await anext(iterator)
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/utils.py", line 850, in __anext__
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/anyio/to_thread.py", line 63, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2502, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 986, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/utils.py", line 833, in run_sync_iterator_async
return next(iterator)
^^^^^^^^^^^^^^
File "/home/user/.local/lib/python3.11/site-packages/gradio/utils.py", line 1017, in gen_wrapper
response = next(iterator)
^^^^^^^^^^^^^^
File "/home/user/app/acestep/gradio_ui/events/__init__.py", line 671, in generation_wrapper
raise gr.Error(f"Failed to create sample: {result.status_message}")
gradio.exceptions.Error: "Failed to create sample: ❌ Error generating from formatted prompt: CUDA error: an illegal memory access was encountered\nSearch for cudaErrorIllegalAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1\nCompile with TORCH_USE_CUDA_DSA` to enable device-side assertions.\n"
2026-01-30 22:48:25.567 | INFO | acestep.inference:generate_music:377 - [generate_music] LLM usage decision: thinking=True, use_cot_caption=True, use_cot_language=True, use_cot_metas=True, need_lm_for_cot=True, llm_initialized=True, use_lm=True
2026-01-30 22:48:25.567 | INFO | acestep.inference:generate_music:435 - LM chunk 1/1 (infer_type=llm_dit) (size: 2, seeds: [1396977744, 3121800500])
2026-01-30 22:48:25.567 | INFO | acestep.llm_inference:generate_with_stop_condition:897 - Batch Phase 1: Generating CoT metadata (once for all items)...
2026-01-30 22:48:25.569 | INFO | acestep.llm_inference:generate_with_stop_condition:905 - generate_with_stop_condition: formatted_prompt=<|im_start|>system

Instruction

Generate audio semantic tokens based on the given conditions:

<|im_end|>

ACE-Step org

Thanks for the log and for reporting this issue. This bug is a known one, but I haven’t been able to identify the root cause yet—it can’t be reproduced locally. It may be related to concurrency and environment configurations. I’ll keep digging into it.

ChuxiJ changed discussion status to closed

Sign up or log in to comment