feat: asr & tts & main - #1206
Merged
Merged
Conversation
…t case (#1133) * feat: add asr guarder * chore: refine code * feat: optimize Azure ASR integration test with comprehensive validation and improved error handling * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io>
* feat: add asr guarder * chore: refine code * feat: refine azure asr extension * feat: refine azure asr extension * feat: add vscode settings * feat: add configs for test * feat: refine azure asr extension * feat: refine azure asr extension * feat: refine azure asr extension * feat:add azure_asr_python extension connect and reconnect guarder test case (#1133) * feat: add asr guarder * chore: refine code * feat: optimize Azure ASR integration test with comprehensive validation and improved error handling * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * feat: refine azure asr extension * feat: refine asr guarder * test comment * feat: add ASR vendor error detection test and improve Python formatting - Add comprehensive test for ASR vendor error detection (test_vendor_error.py) - Validates error format according to ASR protocol specification - Tests error code categorization and session ID consistency - Includes vendor_info validation and metadata handling - Provides detailed error validation and test summary - Update VSCode settings for Python development - Set Black formatter as default for Python files - Improve code formatting consistency * fix: update error code validation to require NON_FATAL_ERROR - Change _validate_error_code_types to require error code = NON_FATAL_ERROR (1000) - Update error message to use NON_FATAL_ERROR constant for clarity - Update log message to indicate the validation rule * fix: add type annotations to resolve type checking errors - Add type annotations for error_code, vendor_info, metadata, and session_id - Add type annotations for method parameters json_data - Fix all type checking errors in test_vendor_error.py --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io>
…#1142) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test --------- Co-authored-by: zhangpeng <zhangpeng@agora.io>
* feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output --------- Co-authored-by: zhangpeng <zhangpeng@agora.io>
* feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn>
…st files (#1155) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn>
…guarder test (#1157) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files * fix: move dump and dump_path from params to config root level in asr_guarder test --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn>
* feat: add new deepgram * feat: add new deepgram * feat: add new deepgram * fix: asr_deepgram websockets * feat: add new deepgram * feat: add new deepgram * feat: add new deepgram * fix: asr_deepgram websockets * fix: handle_finalize_api * fix: asr deepgram await final --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com>
* feat(tts_guarder): send a text to verify the correctness of the APIKey * feat: add a unit test task that does not require installation for extension in Taskfile.yml * feat(unittest): update unit test
Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com>
Co-authored-by: chenyuguo <chenyuguo@agora.io>
Co-authored-by: chenyuguo <chenyuguo@agora.io>
* fix: 11labs tts loop maintance websocket connection Description: 11labs tts use loop to maintance websocket connetion. fix google tts ut unstable issue format code * test: modify google tts ut * fix: format file --------- Co-authored-by: wangyimin <wangyimin@agora.io>
* feat: add Google ASR extension unit test cases and configuration updates * feat: implement long duration ASR streaming with auto-restart mechanism - Add stream_max_duration config (270s) to prevent Google ASR 5-minute timeout - Implement automatic stream restart in google_asr_client.py - Add long duration integration test (test_long_duration_stream.py) - Modify test startup script to skip long duration tests by default - Fix Google ASR 409 Max duration error for extended streaming sessions * feat: optimize bytedance ASR extension with audio buffering and connection management - Add AudioBufferManager for 200ms audio chunking (6400 bytes) - Implement WebSocket keepalive with 30s ping interval - Add automatic reconnection with exponential backoff - Optimize connection state checking and error handling - Clean up debug logs and improve code readability - Follow Bytedance official streaming ASR recommendations * fix: resolve code quality issues - Remove unused imports (time, FINALIZE_MODE_DISCONNECT, FINALIZE_MODE_MUTE_PKG) - Fix f-string without variable interpolation - Fix attribute definition issue by moving _last_logged_state to __init__ method - Optimize code formatting and remove unnecessary line breaks * fix: improve bytedance ASR extension error handling and logging * revert: restore property.json to original state * chore: update manifest.json files for ASR extensions --------- Co-authored-by: PaulZhang <zhangpeng@agora.io>
* fix: xfyun timeout * fix: xfyun timeout --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com>
* fix: bytedance reconnect * fix: bytedance tts loop maintance websocket connection --------- Co-authored-by: chenyuguo <chenyuguo@agora.io> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com>
* fix: language * fix: language_utils --------- Co-authored-by: liaochenliang <liaochenliang@agora.io>
* fix: fix wrong dump file naming convention in OpenaiTTSExtension * chore: update version to 0.1.3 in openai_tts2_python manifest
…1296) Co-authored-by: wangyimin <wangyimin@agora.io>
* feat: add azure_tts_python * fix: lint code * chore: format code * chore: lint field_validator
* fix: mute_pkg * fix: format * fix: finalize_mode * fix: mute_pkg --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com>
plutoless
marked this pull request as ready for review
August 20, 2025 19:09
BenWeekes
pushed a commit
that referenced
this pull request
Nov 12, 2025
* feat: add asr guarder * chore: refine code * feat: refine azure asr extension * feat: refine azure asr extension * feat: add vscode settings * feat: add configs for test * feat: refine azure asr extension * feat: refine azure asr extension * feat: refine azure asr extension * feat:add azure_asr_python extension connect and reconnect guarder test case (#1133) * feat: add asr guarder * chore: refine code * feat: optimize Azure ASR integration test with comprehensive validation and improved error handling * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * feat: refine azure asr extension * feat: refine asr guarder * feat: refine asr guarder * feat: refine asr guarder workflow * fix: requirements * fix: workflow * fix: workflow * feat: add comprehensive ASR vendor error validation (#1137) * feat: add asr guarder * chore: refine code * feat: refine azure asr extension * feat: refine azure asr extension * feat: add vscode settings * feat: add configs for test * feat: refine azure asr extension * feat: refine azure asr extension * feat: refine azure asr extension * feat:add azure_asr_python extension connect and reconnect guarder test case (#1133) * feat: add asr guarder * chore: refine code * feat: optimize Azure ASR integration test with comprehensive validation and improved error handling * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * feat: refine azure asr extension * feat: refine asr guarder * test comment * feat: add ASR vendor error detection test and improve Python formatting - Add comprehensive test for ASR vendor error detection (test_vendor_error.py) - Validates error format according to ASR protocol specification - Tests error code categorization and session ID consistency - Includes vendor_info validation and metadata handling - Provides detailed error validation and test summary - Update VSCode settings for Python development - Set Black formatter as default for Python files - Improve code formatting consistency * fix: update error code validation to require NON_FATAL_ERROR - Change _validate_error_code_types to require error code = NON_FATAL_ERROR (1000) - Update error message to use NON_FATAL_ERROR constant for clarity - Update log message to indicate the validation rule * fix: add type annotations to resolve type checking errors - Add type annotations for error_code, vendor_info, metadata, and session_id - Add type annotations for method parameters json_data - Fix all type checking errors in test_vendor_error.py --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * fix: workflow * fix: asr guarder * fix: asr guarder * feat: add comprehensive ASR testing suite with multi-language support (#1142) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> * feat: add comprehensive ASR integration test suite (#1148) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> * fix: azure asr * feat: improve ASR test validation logicFeat/asr guarder (#1152) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> * fix: asr guarder task file * refactor: replace silence packet logic with finalize signal in ASR test files (#1155) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> * feat: add tencent_asr_python extension (#1154) * fix: move dump and dump_path from params to config root level in asr_guarder test (#1157) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files * fix: move dump and dump_path from params to config root level in asr_guarder test --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> * feat: add new deepgram (#1151) * feat: add new deepgram * feat: add new deepgram * feat: add new deepgram * fix: asr_deepgram websockets * feat: add new deepgram * feat: add new deepgram * feat: add new deepgram * fix: asr_deepgram websockets * fix: handle_finalize_api * fix: asr deepgram await final --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * fix: asr guarder workflow * fix: asr guarder workflow * feat: add new aliyun bigmodel asr (#1166) * feat: add new task aliyun_asr_bigmodel_python * fix: format aliyun_asr_bigmodel_python --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * feat: add new bytedance asr (#1170) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files * fix: move dump and dump_path from params to config root level in asr_guarder test * refactor: improve code quality and extract config - Extract BytedanceASRConfig to config.py - Create const.py for error codes and constants - Remove debug code and fix inheritance issues - Add configurable workflow and reconnection params - Update tests and improve code organization * cleanup: remove unused test config files - Remove property_connection_error.json - Remove property_network_error.json - Remove property_reconnectable_error.json * refactor: Convert Chinese comments to English in bytedance_asr extension - Update error code comments in const.py from Chinese to English - Update finalize logic comments in extension.py from Chinese to English - Update protocol description comments in bytedance_asr.py from Chinese to English This change improves code readability and maintains consistency with English commenting standards. * test: Update asr guarder test files and bytedance_asr extension - Update test files in asr_guarder integration tests: - test_asr_finalize.py: Enhanced finalize testing functionality - test_dump.py: Improved dump testing capabilities - test_metrics.py: Updated metrics validation logic - Update bytedance_asr extension.py with bug fixes and improvements These changes improve test coverage and reliability for ByteDance ASR integration testing. * style: Change comments from Chinese to English in asr_guarder tests --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * feat: update ten_ai_base version and image version * fix: add words in asr result * Feat/asr guarder (#1179) * feat: tts2 + bytedance duplex tts * feat: initial bytedance tts duplex support * fix: prevent empty text request * feat: ten_ai_base upgrade to 0.6.11 * fix: fix format & lint * Feat/tts2 interface (#1117) * feat: adapt to interface * fix: adapt to new tts logic * fix: revert ten_ai_base * fix: adapt tts to ten_ai_base 6.0.12 * feat:support ttfb & dump * feat: update ten_ai_base to 0.6.13 * fix: update deps & fix lint/format * feat: upgrade ten_ai_base to 0.6.15 * feat: support config.params * fix: separate config.py * fix: adjust connection logic * fix: fix logs * feat: support config log print with masking * feat: add tts_audio_start & tts_audio_end, fix bugs * feat:upgrade ten_ai_base & fix format * fix: fix gladia test * fix: adapt tests * fix: fix format * fix: fix asr tests * feat: fix format * fix: fix asr test * fix: fix format * fix: fix tests * ut: remove unused ut --------- Co-authored-by: plutoless <zhangqianze@agora.io> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * fix: optimize audio timestamp test for multiple ASR providers compatibility (#1181) * feat: add asr guarder * chore: refine code * feat: refine azure asr extension * feat: refine azure asr extension * feat: add vscode settings * feat: add configs for test * feat: refine azure asr extension * feat: refine azure asr extension * feat: refine azure asr extension * feat:add azure_asr_python extension connect and reconnect guarder test case (#1133) * feat: add asr guarder * chore: refine code * feat: optimize Azure ASR integration test with comprehensive validation and improved error handling * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * feat: refine azure asr extension * feat: refine asr guarder * feat: refine asr guarder * feat: refine asr guarder workflow * fix: requirements * fix: workflow * fix: workflow * feat: add comprehensive ASR vendor error validation (#1137) * feat: add asr guarder * chore: refine code * feat: refine azure asr extension * feat: refine azure asr extension * feat: add vscode settings * feat: add configs for test * feat: refine azure asr extension * feat: refine azure asr extension * feat: refine azure asr extension * feat:add azure_asr_python extension connect and reconnect guarder test case (#1133) * feat: add asr guarder * chore: refine code * feat: optimize Azure ASR integration test with comprehensive validation and improved error handling * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms * feat: Add Azure ASR reconnection test, enhance error handling and validation mechanisms --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * feat: refine azure asr extension * feat: refine asr guarder * test comment * feat: add ASR vendor error detection test and improve Python formatting - Add comprehensive test for ASR vendor error detection (test_vendor_error.py) - Validates error format according to ASR protocol specification - Tests error code categorization and session ID consistency - Includes vendor_info validation and metadata handling - Provides detailed error validation and test summary - Update VSCode settings for Python development - Set Black formatter as default for Python files - Improve code formatting consistency * fix: update error code validation to require NON_FATAL_ERROR - Change _validate_error_code_types to require error code = NON_FATAL_ERROR (1000) - Update error message to use NON_FATAL_ERROR constant for clarity - Update log message to indicate the validation rule * fix: add type annotations to resolve type checking errors - Add type annotations for error_code, vendor_info, metadata, and session_id - Add type annotations for method parameters json_data - Fix all type checking errors in test_vendor_error.py --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> * fix: workflow * fix: asr guarder * fix: asr guarder * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * feat: add comprehensive ASR testing suite with multi-language support (#1142) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * feat: add comprehensive ASR integration test suite (#1148) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> * fix: azure asr * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * feat: improve ASR test validation logicFeat/asr guarder (#1152) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> * fix: asr guarder task file * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files * refactor: replace silence packet logic with finalize signal in ASR test files (#1155) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> * feat: add tencent_asr_python extension (#1154) * fix: move dump and dump_path from params to config root level in asr_guarder test * fix: move dump and dump_path from params to config root level in asr_guarder test (#1157) * feat: add comprehensive ASR testing suite with multi-language support * feat: Add audio clock ASR test class to verify the clock field of ASR results * feat: remove text content validation from ASR tests and add timeout to finalize test * refactor: remove non-final result validation from test_asr_result.py - Remove non-final results tracking and validation - Delete _validate_non_final_and_final_results method - Delete _validate_data_format_consistency method - Delete _validate_id_consistency method - Add _validate_final_result_only method for final result only - Update on_data to only process final results - Remove non-final result storage and validation logic - Update test description to remove non-final result requirements * feat: add comprehensive ASR integration test suite Add new test cases: - test_asr_finalize.py: Validate ASR finalize functionality - test_dump.py: Validate ASR dump functionality with real audio file - test_metrics.py: Validate ASR metrics (TTFW, TTLW) Update existing test cases with test case headers: - test_audio_timestamp.py: Add timestamp validation test header - test_connection_timing.py: Add connection timing test header - test_multi_language.py: Add multi-language test header - test_reconnection.py: Add reconnection test header - test_vendor_error.py: Add vendor error test header Features: - Add clear test case identification with headers and descriptions - Implement real audio file usage in dump test - Add comprehensive validation for ASR finalize mechanism - Add metrics validation for TTFW and TTLW - Improve test readability with emoji and structured output * fix: azure asr * feat: implement ASR result ID grouping management - Create id_group_manager.py to support group-based ASR result management - Change audio filename from 16k_en_us_helloworld.pcm to 16k_en_us.pcm - Support multiple ASR result groups, each containing multiple non-final results and one final result - Maintain consistent IDs within each group, different IDs between groups * feat: improve ASR test validation logic - Remove hardcoded vendor validation in test_metrics.py - Implement dynamic .pcm file detection in dump tests - Add ID group management for ASR results in test_asr_finalize.py - Support multiple ASR result groups with consistent IDs within each group - Remove hardcoded PCM filenames, use glob pattern to find any .pcm file - Make tests more flexible for different ASR extensions and vendors - Add 16k_en_us.pcm audio file for testing * chore: update .gitignore to allow test_data files * refactor: clean up unused logic and comments, improve ASR result test case * refactor: replace silence packet logic with finalize signal in ASR test files * fix: move dump and dump_path from params to config root level in asr_guarder test --------- Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: sunxilin <sunxilin@shengwang.cn> * feat: add new deepgram (#1151) * feat: add new deepgram * feat: add new deepgram * feat: add new deepgram * fix: asr_deepgram websockets * feat: add new deepgram * feat: add new deepgram * feat: add new deepgram * fix: asr_deepgram websockets * fix: handle_finalize_api * fix: asr deepgram await final --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * refactor: improve code quality and extract config - Extract BytedanceASRConfig to config.py - Create const.py for error codes and constants - Remove debug code and fix inheritance issues - Add configurable workflow and reconnection params - Update tests and improve code organization * cleanup: remove unused test config files - Remove property_connection_error.json - Remove property_network_error.json - Remove property_reconnectable_error.json * fix: asr guarder workflow * fix: asr guarder workflow * feat: add new aliyun bigmodel asr (#1166) * feat: add new task aliyun_asr_bigmodel_python * fix: format aliyun_asr_bigmodel_python --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * refactor: Convert Chinese comments to English in bytedance_asr extension - Update error code comments in const.py from Chinese to English - Update finalize logic comments in extension.py from Chinese to English - Update protocol description comments in bytedance_asr.py from Chinese to English This change improves code readability and maintains consistency with English commenting standards. * test: Update asr guarder test files and bytedance_asr extension - Update test files in asr_guarder integration tests: - test_asr_finalize.py: Enhanced finalize testing functionality - test_dump.py: Improved dump testing capabilities - test_metrics.py: Updated metrics validation logic - Update bytedance_asr extension.py with bug fixes and improvements These changes improve test coverage and reliability for ByteDance ASR integration testing. * style: Change comments from Chinese to English in asr_guarder tests * fix: optimize audio timestamp test for multiple ASR providers compatibility * chore: update asr guarder workflow with latest image version and env config --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: Jachin <ygcaicn@gmail.com> Co-authored-by: liaochenliang <120015237+liaochenliang@users.noreply.github.com> Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * fix: update ten_ai_base version to 0.6 * chore: azure asr version update * feat: add openai_asr_python extension (#1176) * chore: ten_ai_base version * fix: tencent asr version * [feat]tts2 (#1183) * [feat]tts2 * [feat]UT test * [Fix]manifest * [Fix]manifest --------- Co-authored-by: chenyuguo <chenyuguo@agora.io> Co-authored-by: pen <chenyuguo1999@163.com> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * feat: add Tencent TTS Python extension (#1199) Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * fix: azure asr * feature: main pipeline (#1134) * feat: implement main_control extension * fix: add tests and make it run * test: smoke * feat: add a example graph * feat: simplify main_control implementation * feat: build voice_assistant_star graph * fix: 0.10 conversion * feat: simplify further * feat: use ten selector * test: delete main_control unit tests The tester will not receive messages with set_dest, so we can't observe and assert. * feat: make greeting configurable * fix: fix playground * fix: align ten_ai_base version * fix: refactor code * fix: finalize code * feat: initial llm_exec * feat: llm2 * feat: using llm2 * feat: add llm_exec & implement interrupt * fix: fix formats * fix: fix lint issues * fix: adapting tools * fix: fix asr interfaces * feat: updates for tools * feat: llm updates * feat: support function calls * feat: adapt message channel * fix: do not hard limit ten_ai_base version * fix: do not lock ten_ai_base version * fix: fix azure version & ten_ai_base version * fix: fix bugs --------- Co-authored-by: jun <875241499@qq.com> * fix: temp fix for task use error * fix: remove unnecessary connections & fix transcript * fix: enable tts2 & fix llm tool * feat: add aws_asr_python extension (#1200) * feat: add aws_asr_python extension * chore: refine aws_asr_python extension --------- Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * feat: add soniox asr extension (#1201) * feat: soniox baseline implementation * test: basic framework * test: basic mocked test * feat: pass guarder tests * test: add unit tests --------- Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * feat: add xfyun asr extension (#1202) * feat: add new xfyun asr * feat: add new xfyun bigmodel asr * fix: _handle_asr_result * fix: _handle_asr_result * fix: websockets * feat: Add xfyun asr dialect python * fix: DUMP_FILE_NAME * fix: tests config * fix: interface json --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * fix: adapt with latest ten_ai_base * fix: fix metadata * Feat/main plus (#1221) * fix: refactor main control, added agent file * feat: upgrade ten build image & refactor main pipeline code * feat: place main into default folder * feat: rename main_python & remove module selector/property editor in playground * feat: support mllm initial draft * feat: support realtime conversation * feat: support realtime api toolcall + fix web ui encoding issue * fix: remove tmp path * feat: add lint-extension task * fix: azure asr lint * feat: adjust openai mllm protocols * test: guarder test for tts and fix bug in bydance_tts_duplex (#1187) * test: guarder test for tts and fix bug in bydance_tts_duplex * test: change some info in tts guarder test * test: improve guarder test --------- Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * feat: upgrade ten_ai_base to 0.6 latest * feat: support grouped names for connections * fix: fix connections * fix: audio dumper in azure asr * fix: xfyun asr lint (#1237) * fix: xfyun asr lint * fix: dump * fix: aliyun asr * fix: deepgram asr lint --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> * chore: lint code (#1235) * fix: tencent_asr_python lint code * fix: aws_asr_python lint code * fix: openai_asr_python lint code * feat: add new dify/coze extensions * feat: add new gemini v2v & stepfun v2v extensions * Update ai_agents.yaml * Update ai_agents.yaml * feat: fix docker file * fix: refactor azure mllm * [feat]tts guarder test (#1240) Co-authored-by: chenyuguo <chenyuguo@agora.io> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * feat: add aws polly tts extension (#1233) * feat: add aws polly tts extension * fix: fix issue for guarder test * feat: bytedance_tts_duplex bump version to 0.1.1 * feat: add JSON ADC credentials for Google ASR and streamline logs; fix Bytedance ASR config lint (#1243) * feat: restore google_asr_python extension with ADC authentication support * fix: correct imports, reconnection handling, logging verbosity, and test PYTHONPATH * feat: support ADC credentials from JSON string and streamline logs - Add adc_credentials_string config and GOOGLE_APPLICATION_CREDENTIALS_STRING env variable\n- Priority: file path > JSON string > default ADC\n- Reduce noisy logs (move verbose details to debug), keep key lifecycle info at info * fix: simplify gRPC error handling by using str(e); raise-from for invalid JSON; use RuntimeError on init validation * fix: avoid FieldInfo no-member by copying params to dict before encryption * chore: update manifest metadata --------- Co-authored-by: PaulZhang <zhangpeng@agora.io> * fix: asr extension lint fix * Feat/tts refactor-elvenlabs (#1220) * feat: 11labs refactor and change annotation in guarder test to english * fix: fix manifest.json issue * feat: add back glm mllm ext * feat: remove redundant extensions * fix: fix manifest.json * feat: adjust examples * fix: google asr * fix: format fix * fix: fix lint issues * fix: fix lint issues * fix: adjust install script to take out llama deps * fix: aws_asr_python unitest (#1247) * fix: aws_asr_python unitest * fix: openai_asr_python, tencent_asr_python unitest --------- Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * fix: add format for main * chore: update version for aws openai and tencent asr * fix: replace all base_uri with base_url * feat: support multiple example local packages * feat: adjust demo structure * fix: fix reasoning mode * fix: tts manifest (#1251) * fix: tts manifest * fix: tts manifest --------- Co-authored-by: chenyuguo <chenyuguo@agora.io> * fix: amazon-transcribe version (#1249) * fix: amazon-transcribe version * chore: update ext version * feat: adding demo cases.. * feat: google tts * fix: add gemini live * fix: fix lint issues * fix: fix gladia tests * fix: fix tests * feat: add rest demo graphs * feat: demo update * fix: UT for aliyun gladia speechmatics * fix: format update * feat: add cosy TTS Python extension (#1252) * feat: add Cosy TTS configuration and enhance Tencent TTS error handling * feat: implement Cosy TTS client and enhance audio processing capabilities * chore: update Tencent TTS manifest and improve logging messages * refactor: update type hints in Cosy TTS client and rename close method to stop; add initial test files for audio dump and text input end logic * feat: add error handling for TTS failures and implement tests for invalid parameters * feat: add unit tests for TTFB metrics in Cosy TTS extension * feat: add unit test for parameter passthrough in Cosy TTS client * feat: add robustness test for handling connection drops in Cosy TTS extension * refactor: rename close method to stop in Tencent TTS client and update related tests for audio flushing and text input end logic * feat: enhance error handling in Tencent TTS extension for authorization failures and improve test coverage for invalid parameters * fix: increase timeout for receiving messages in Cosy TTS client and clarify test setup for invalid parameters * feat: implement metrics testing for TTFB and audio frame reception in Tencent TTS extension * chore: remove unnecessary blank lines in test files for Cosy and Tencent TTS extensions * feat: implement parameter passthrough testing for Tencent TTS extension, ensuring custom parameters are correctly forwarded to the client * feat: add robustness testing for connection recovery in Tencent TTS extension, ensuring proper handling of connection drops and successful reconnection * feat: add black list parameter handling in Cosy and Tencent TTS configurations to improve sensitive data management * feat: enhance error handling in Tencent TTS extension by adding support for invalid parameters and updating related error logging * feat: enhance Tencent TTS extension with PCMWriter management and flush handling * refactor: streamline PCMWriter management in Tencent TTS extension by introducing dedicated cleanup and management methods * chore: add ModuleType import to Tencent TTS extension for improved module management * feat: enhance Tencent TTS extension by adding TTS text result handling and improving audio end processing * feat: improve logging and request handling in Tencent TTS extension by adding current request ID tracking and refining error message handling * feat: enhance Cosy TTS extension with improved PCMWriter management, refined logging, and request flushing capabilities * refactor: remove unused pcm_dump_file attribute from Tencent TTS extension to streamline code * chore: update pylint configuration to ignore specific paths in Tencent TTS extension * refactor: add comments for future enhancements and streamline parameter checks in Cosy and Tencent TTS configurations * refactor: implement dynamic audio format selection in CosyTTSClient * refactor: streamline flush handling in Cosy and Tencent TTS extensions * refactor: enhance logging for empty text requests in Cosy and Tencent TTS extensions * refactor: improve code readability with consistent formatting in Cosy and Tencent TTS extensions * refactor: improve exception handling in CosyTTSClient by chaining exceptions --------- Co-authored-by: Hugo Chan <47882165+HugoChaan@users.noreply.github.com> * fix: tts error (#1253) * fix: tts error * fix: tts error --------- Co-authored-by: chenyuguo <chenyuguo@agora.io> * fix: 11labs ut use fark key instead of reading environment var * fix: fix google tts issue and update tencent tts version * test: tts guarder test change Description: don't check sample rate difference if vendor not support don't check tts text result since it is only used for enable_words no error msg when request after request end true * fix: fix google asr test * fix: fix format * fix: fix ut tests * fix: fix google tts ut issue * feat: fix format * fix: remove dependency on pyaudio * fix: remove all sounddevice dependencies * fix: fix openai_asr_python tests * fix: fix tests * fix: fix format * fix: avoid error racing condition * fix: codeQL warning * fix: codeQL warning * fix: fix flush behavior * fix: fix format * feat: update 11labs tts and google tts version Description: update latest 11labs for ten store add python package version for google tts and update for ten store * fix:tts flush (#1260) Co-authored-by: chenyuguo <chenyuguo@agora.io> * feat: speechmatics (#1261) * feat: speechmatics * fix: task format --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> * fix: add package include list for asr exts and upgrade * feat: update package manifest * fix: minimax flush (#1265) Co-authored-by: chenyuguo <chenyuguo@agora.io> * fix: fix interrupt issues * feat: add Cartesia TTS、Openai TTS and Fish Audio TTS Extenions (#1268) * feat(tts): add cartesia tts * feat(tts): add openai tts * feat(tts): add fish audio tts * chore(style): format code * fix(tests): increase wait time for TTS output in invalid text handling test * fix(lint): fix lint error * feat: support spanish in speechmatics * feat: update package include for fish_audio openai and cartesia tts * Fix property for minimax tts * fix: minimax flush * fix: property.json * fix: lint and format --------- Co-authored-by: chenyuguo <chenyuguo@agora.io> * fix: fix lint issue * fix: final (#1278) Co-authored-by: liaochenliang <liaochenliang@agora.io> * fix: fix main cascade * fix: fix cascade * feat: implement tts google bytdance 11labs (#1279) * feat: improve tts bytedance google and 11labs Description: implement thress tts extension as new flush behavior update package version and included folders or files as relase * feat: update property behavior for tts google 11labs and bytedance duplex * test: add sleep to make case stable * fix: fix integration test err err and unit test err (#1280) * feat(tts_guarder): send a text to verify the correctness of the APIKey * feat: add a unit test task that does not require installation for extension in Taskfile.yml * feat(unittest): update unit test * fix: add polly_tts property.json (#1282) Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * fix:self test bugs (#1283) Co-authored-by: chenyuguo <chenyuguo@agora.io> * feat: support turn & reasoning * fix: handling interrupt * fix: fix reasoning & adapt * fix: fix formats * feat: add ten vad use case * fix: bytedance reconnect (#1285) Co-authored-by: chenyuguo <chenyuguo@agora.io> * fix: 11labs tts loop maintance websocket connection (#1289) * fix: 11labs tts loop maintance websocket connection Description: 11labs tts use loop to maintance websocket connetion. fix google tts ut unstable issue format code * test: modify google tts ut * fix: format file --------- Co-authored-by: wangyimin <wangyimin@agora.io> * fix: fix bytedance asr finalize (#1288) * feat: add Google ASR extension unit test cases and configuration updates * feat: implement long duration ASR streaming with auto-restart mechanism - Add stream_max_duration config (270s) to prevent Google ASR 5-minute timeout - Implement automatic stream restart in google_asr_client.py - Add long duration integration test (test_long_duration_stream.py) - Modify test startup script to skip long duration tests by default - Fix Google ASR 409 Max duration error for extended streaming sessions * feat: optimize bytedance ASR extension with audio buffering and connection management - Add AudioBufferManager for 200ms audio chunking (6400 bytes) - Implement WebSocket keepalive with 30s ping interval - Add automatic reconnection with exponential backoff - Optimize connection state checking and error handling - Clean up debug logs and improve code readability - Follow Bytedance official streaming ASR recommendations * fix: resolve code quality issues - Remove unused imports (time, FINALIZE_MODE_DISCONNECT, FINALIZE_MODE_MUTE_PKG) - Fix f-string without variable interpolation - Fix attribute definition issue by moving _last_logged_state to __init__ method - Optimize code formatting and remove unnecessary line breaks * fix: improve bytedance ASR extension error handling and logging * revert: restore property.json to original state * chore: update manifest.json files for ASR extensions --------- Co-authored-by: PaulZhang <zhangpeng@agora.io> * fix: xfyun timeout (#1286) * fix: xfyun timeout * fix: xfyun timeout --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> * fix: ws_client cancel the receiver_task when ws disconnected (#1290) * chore: update version of openai and tencent asr * Feat/tts2 improve bytedance (#1291) * fix: bytedance reconnect * fix: bytedance tts loop maintance websocket connection --------- Co-authored-by: chenyuguo <chenyuguo@agora.io> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * feat: remove legacy tts * fix: language (#1293) * fix: language * fix: language_utils --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> * fix: fix wrong dump file naming convention in OpenaiTTSExtension (#1297) * fix: fix wrong dump file naming convention in OpenaiTTSExtension * chore: update version to 0.1.3 in openai_tts2_python manifest * fix: fix 11labs tts no audio for short text and change websockt loop (#1296) Co-authored-by: wangyimin <wangyimin@agora.io> * feat: azure tts python (#1294) * feat: add azure_tts_python * fix: lint code * chore: format code * chore: lint field_validator * chore: update version for ten publish * fix: mute_pkg (#1300) * fix: mute_pkg * fix: format * fix: finalize_mode * fix: mute_pkg --------- Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: Ethan Zhang <qianze.zhang@hotmail.com> * fix: update default vendors & fix fish audio issue --------- Co-authored-by: sunxilin <sunxilin@shengwang.cn> Co-authored-by: PaulZhang <zhang.peng.paul@gmail.com> Co-authored-by: zhangpeng <zhangpeng@agora.io> Co-authored-by: Jachin <ygcaicn@gmail.com> Co-authored-by: liaochenliang <120015237+liaochenliang@users.noreply.github.com> Co-authored-by: liaochenliang <liaochenliang@agora.io> Co-authored-by: xxxxl_sun <31622273+sunxilin@users.noreply.github.com> Co-authored-by: Wang Yimin <wym0207@outlook.com> Co-authored-by: Hugo Chan <47882165+HugoChaan@users.noreply.github.com> Co-authored-by: chenyuguo <chenyuguo@agora.io> Co-authored-by: pen <chenyuguo1999@163.com> Co-authored-by: sunshinexcode <24xinhui@163.com> Co-authored-by: jun <875241499@qq.com> Co-authored-by: Jay Zhang <wangyoucao577@gmail.com> Co-authored-by: Xianmeng Tang <tangxianmeng@live.com> Co-authored-by: wangyimin <wangyimin@agora.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.