3 : (Won Jang et al.: Musical Instrument Conversion based Music Ensemble Application Development for Smartphone) (Special Paper) 22 2, (JBE Vol

Size: px

Start display at page:

Download "3 : (Won Jang et al.: Musical Instrument Conversion based Music Ensemble Application Development for Smartphone) (Special Paper) 22 2, (JBE Vol"

호린 위
4 years ago
Views:

1 3 (Special Paper) 22 2, (JBE Vol. 22, No. 2, March 2017) https//doi.org/ /jbe ISSN (Online) ISSN (Print) a), a), a), a) Musical Instrument Conversion based Music Ensemble Application Development for Smartphone Won Jang a), Hyo-Jin Cho a), Seong-Hyeon Shin a), and Hochong Park a)..,.,,,.. Abstract In this paper, we propose a musical instrument conversion based music ensemble application for smartphone. If we try to create ensemble music using virtual instruments provided by the conventional smartphone application, we should know how to play each instrument. In addition, it is impossible to play the instrument in a natural way if the smartphone screen cannot show the entire part of the instrument. To solve this problem, in this paper, we propose a smartphone application that records music sound by playing an acoustic guitar, converts it to other instruments' sound, applies effect to the converted sound, and creates a final ensemble music after mixing all sounds. Using the proposed application, the user can create ensemble music only by playing the acoustic guitar. Keyword music ensemble, smartphone application, instrument conversion, mixing a) (Dept. of Electronics Engineering, Kwangwoon University) Corresponding Author (Hochong Park) hcpark@kw.ac.kr Tel ORCID http//orcid.org/ Manuscript received January 10, 2017; Revised March 9, 2017; Accepted March 9, Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND (http//creativecommons.org/licenses/by-nc-nd/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 (JBE Vol. 22, No. 2, March 2017). (music ensemble).. [1]... (mixer),... Garage Band,,,,..,.,,..,..,,. (effector)....,,. (distortion), (compressor).,,, Fig. 1. Spectrum of acoustic guitar sound and electric guitar sound

3 2. Fig. 2. Equalizer for converting acoustic guitar sound to electric guitar sound.. 1. 2. 2. 6 4 4 1. 4, 1.,, (time stretching) [2]. 3 SOLA (synchronous overlap and add). (a) 3. SOLA (a). (b) Fig.

3 3 2. Fig. 2. Equalizer for converting acoustic guitar sound to electric guitar sound , 1.,, (time stretching) [2]. 3 SOLA (synchronous overlap and add). (a) 3. SOLA (a). (b) Fig. 3. Process for pitch shifting of music source using SOLA (a) Computation of cross-correlation and block synchronization. (b) Crossfading and overlap-add (b) 4. Fig. 4. Spectrum modification by pitch shifting

(JBE Vol. 22, No. 2, March 2017) 3(a) a, 3(b) [3].,. 4. 1 2. SOLA a 0.5, 2 2. 5. 6. 3. 5. Fig. 5. Spectrum of pitch-shifted acoustic guitar sound and bass sound 6.

4 (JBE Vol. 22, No. 2, March 2017) 3(a) a, 3(b) [3]., SOLA a 0.5, Fig. 5. Spectrum of pitch-shifted acoustic guitar sound and bass sound 6. Fig. 6. Equalizer for converting acoustic guitar sound to bass sound 7. Fig. 7. Spectrum of acoustic guitar sound and keyboard sound

5 3 8. Fig. 8. Equalizer for converting acoustic guitar sound to keyboard sound , (reverberation). 9 [4]..., FFT (fast Fourier transform). 5.2 (delay)., Fig. 9. Example of room impulse response for reverberation effect

6 (JBE Vol. 22, No. 2, March 2017) 10. Fig. 10. Example of room impulse response for delay effect [5]. 5.3, (overdrive),. [6].., (1), C. C (a) db, C., [6]., (2).. 11(b) db. f or f or f or., db. 11(c) db. (compressor threshold, CT) -30dB 1 (compressor slope, CS). (a) (b) (c) 11. (a). (b). (c) Fig. 11. Input/output characteristic graph of nonlinear processing effects (a) Distortion. (b) Overdrive. (c) Compressor

7 3 6. db OS Java eclipse. 12.,.,., 12. Fig. 12. Flowchart of the proposed music ensemble application., (a).,,.,,,,,.,.,.,., (metronome),. 13(b).., BPM (bit per minute).. BPM 200BPM. 13(c). Convert.,,,.,. 13(d)., Mix 48,000Hz PCM,.

8 (JBE Vol. 22, No. 2, March 2017) 13. (a). (b). (c). (d) Fig. 13. Example screen of developed smartphone application (a) Screen for function selection. (b) Screen for sound record. (c) Screen for sound conversion. (d) Screen for sound mixing. 14..,., (a). (b). (c) Fig. 14. Spectrum of converted sound and real sound for each instrume (a) Electric guitar. (b) Bass. (c) Keyboard.,.....

장원 외 인 스마트폰을 위한 악기 변환 기반의 음악 합주 애플리케이션 개발 181 3 참 고 문 헌 (References) [1] [2] [3] [4] Ki-Jun Kim, Chang-Ho Myeong and Hochong Park, "Implementation of stereophonic sound system using multiple

Zölzer, "High quality time-domain pitch shifting using PSOLA and transient preservation," Proceeding of Audio Engineering Society Convention, San Francisco, US, pp. 238-241, November 2010.

Jaeyeun Yoon, Junsun Park and Yongok Jin, "A study on the improvement of reverberation characteristics using tapped and nested-allpass delay line," Journal of Broadcast Engineering, Vol.12, No.1, pp.

Udo Zölzer, DAFX Digital Audio Effects, John Wiley and Sons, 2011. 저자소개 장원 년 월 광운대학교 전자공학과 공학사 년 월 현재 광운대학교 전자공학과 석사과정 주관심분야 오디오 음성 신호처리 딥 러닝 - 2017 2-2017 3 ~ - ORCID http//orcid.

9 장원 외 인 스마트폰을 위한 악기 변환 기반의 음악 합주 애플리케이션 개발 참 고 문 헌 (References) [1] [2] [3] [4] Ki-Jun Kim, Chang-Ho Myeong and Hochong Park, "Implementation of stereophonic sound system using multiple smartphones," Journal of Broadcast Engineering, Vol.19, No.6, pp , November A. von dem Kneseback, P. Ziraksaz and U. Zölzer, "High quality time-domain pitch shifting using PSOLA and transient preservation," Proceeding of Audio Engineering Society Convention, San Francisco, US, pp , November Don Henja and Bruce R. Musicus, The SOLAFS time-scale mod- [5] [6] ification algorithm, Technical report, BBN, Jul Jaeyeun Yoon, Junsun Park and Yongok Jin, "A study on the improvement of reverberation characteristics using tapped and nested-allpass delay line," Journal of Broadcast Engineering, Vol.12, No.1, pp , January W. G. Gardner, "Efficient convolution without input-output delay," Journal of Audio Engineering Society, Vol.43, No.3, pp , Udo Zölzer, DAFX Digital Audio Effects, John Wiley and Sons, 저자소개 장원 년 월 광운대학교 전자공학과 공학사 년 월 현재 광운대학교 전자공학과 석사과정 주관심분야 오디오 음성 신호처리 딥 러닝 ~ - ORCID http//orcid.org/ x /, 조효진 년 월 광운대학교 전자공학과 공학사 년 월 현재 광운대학교 전자공학과 석사과정 주관심분야 오디오 음성 신호처리 딥 러닝 ~ - ORCID http//orcid.org/ /, 신성현 년 월 광운대학교 전자공학과 공학사 년 월 현재 광운대학교 전자공학과 석박사통합과정 주관심분야 오디오 음성 신호처리 딥 러닝 ~ - ORCID http//orcid.org/ /, 박호종 - 년 2월 서울대학교 전자공학과 공학사 년 12월 Univ. of Wisconsin-Madison 공학석사 년 5월 Univ. of Wisconsin-Madison 공학박사 년 9월 ~ 1997년 8월 삼성전자 선임연구원 년 9월 ~ 현재 광운대학교 전자공학과 교수 http//orcid.org/ 주관심분야 오디오/음성 신호처리, 3D 오디오, 음악정보처리 ORCID

08김현휘_ok.hwp

08김현휘_ok.hwp (Regular Paper) 21 3, 2016 5 (JBE Vol. 21, No. 3, May 2016) http://dx.doi.org/10.5909/jbe.2016.21.3.369 ISSN 2287-9137 (Online) ISSN 1226-7953 (Print) a), a) An Audio Coding Technique Employing the Inter-channel