Mobile Biometrics 2015. 11. 20 김재희 연세대학교생체인식연구센터 cherup.yonsei.ac.kr
차례 1 생체인식과모바일응용 2 모바일지문인식 3 모바일홍채인식 4 모바일장문인식 5 모바일얼굴인식 6 생체정보의보호 7 맺는말
1. 생체인식과모바일 Biometrics = Bio + Metrics Automatic personal identification using physiological or behavioral traits of a person.
생체인식에서사용되는특징 신체부위 지문 행동습성 서명 홍채 음성 얼굴 장문 Keyboard Dynamics 걸음걸이 변별력 + 지속성
Why Biometrics? Secure Identification by Physical Presence Convenient No need to Carry or Memorize New Solutions Solutions which were not possible before
경제성 : Access Control at Disney
不正, 否認의방지 : Attendance Check
보안 + 보호 : Smart Gun Intelligent Fire Arm, 2003, South Africa
불법입국검출 Use of fingerprint & face recognition Total 1262 detections during 2010. 9 ~ 2011. 12 Detection of illegal entry at immigration
스마트세탁기
모바일생체인식? Needing a handheld or movable identifying solution Police patrol, military, border security, public safety and justice, etc. Ex. Police inspection on a car driver sitting in a car. More recently, smart phones have built-in biometric solutions. For unlocking the phone For on-line payment, etc http://www.datastrip.com/index.html
모바일바이오인식 신체에의한개인식별및인증 실존의보안성 편의성 경제성 [ 전자신문, 2012.9.20. 종합면 p.3] 향후차세대모바일제품에서생체인식의원천기술확보는특허분쟁에대비하기위한최상의방책이며, 또한스마트기기산업전체의중요한과제이다
Multimodal Mobile: HIDE (Morpho)* *http://www.l1id.com For identifying others Iris (640*480 VGA monochrome) Face (640*480 VGA color) Fingerprint (500 dpi)
Datastrip http://www.datastrip.com/index.html
Biometric Engineering Research Center - MMS 2.0 Operating range : 14 ~ 21cm/iris, 25~95cm/face Processing time : less than 1 sec Accuracy : EER of 0.44%/iris, 10.61%/face Size : 15(W) 10(H) 8.3(D)cm3 Weight : 700 g Maximum Enrollments : 3,200,000 persons CPU : Intel 1.2 GHz 4.5 LCD Display Expected Price : $2,000 (Others: $4,000~$6,000)
Multimodal Mobile Biometrics (*http://www.bi2technologies.com) Multimodal Biometric Scanner by smart phone add-on Face Iris Fingerprint Phone Add-on $3000
AOptix Stratus Biometric Scanner* (*http://www.aoptix.com/) Multimodal Biometric Scanner Face Iris Fingerprint Voice iphone Add-on: 2014: $199? (http://www.wptv.com/news/science-tech/aoptix-stratus-biometric-app-foriphone-tech-company-turns-your-phone-into-biometric-scanner)
Those are for identifying others and used by trained person. Unit price and accuracy are more important than user convenience.
They were not so successful! No killer application for them.
Mobile For Phone Unlocking Since 2014 iphone 5S: Touch ID www.apple.com/kr Galuxy S5 http://www.samsung.com/sec/ Pantech Vega: Secrete Note http://www.pantech.co.kr/ Unlocking the phone -> Killer application
List of All Fingerprint Scanner Enabled Smartphones: Now User convenience is more important than price or accuracy for phone unlocking.
What will be the next big application for mobile fingerprint recognition?
What will be the next big application for mobile fingerprint recognition? Mobile E-payment, Which requires accuracy and low cost more than user convenience.
모바일생체인식의필요 Log-in Privacy Mobile banking E-commerce E-health 모바일환경 : 스마트폰, 웨어러블기기, Touch-pad, Tablet PC, Lap-top, 등 패턴, 비밀번호, PKI 인증 망각 유출 사용부인 현모바일인증의문제점해소
2. 모바일지문인식 획득된지문영상 처리된영상 Minutia( 특징점 ): 11 ending points & 17 branches Typically, more than 20-30 minutiae extracted from an optical sensor. More than 10 matched minutiae assure the same fingerprint
영상획득 : 지문입력센서 광학식 : Optical + CCD, 국내대부분기업 : 니트젠, 슈프리마, 디젠트, 유니온컴 장점 : 온도변화에강함, 저가형, 고해상도, 대체로큰지문영상 단점 : 비교적큰사이즈, 센서접촉부의잠상 (latent image)
반도체식 Capacitance : Authen Tec, UPEK, Veridicom, Siemens Thermal: Atmel, 퍼스텍 장점 : 센서의소형화, 양질의이미지, 자체위조방지 ( 일부 ) 단점 : 내구성이떨어짐, 정전기, 취득영상이작음
Swipe Sensor : Semiconductor Thermal : AuthenTec, Atmel, Thomson CSF 장점 : 센서의소형화, 자체위조방지 ( 일부 ), 저가 단점 : 사용불편, 취득영상의질저하와일관성이없음, 인식성능낮음 Area 센서에대한호환성낮음.
초기모바일지문인식 Sensor Size 13.5 3 1.3(mm) (L W H) Resolution (dpi) 500 Sensor Type Swipe Image size (pixel) 192 8 Anti-spoofing protection X
Mobile Fingerprint Scanners iphone 5S: Touch ID www.apple.com/kr 2013.9 Galuxy S5 http://www.samsung.com/sec/ 2014. 2 Pantech Vega: Secrete Note http://www.pantech.co.kr/ 2013.8 www.yonsei.ac.kr
Others, More Recent Sensor at side power button: Sony Xperia Z5 (IFA 2015) Sensor at front touch glass: To be appeared by Crucialtec (MWC 2015)
Sensor size vs # of minutiae 80 70 60 # of minutiae from thumb finger (estimated) 50 40 30 20 Apple: ~5 Solid sensor1: 35 Solid sensor2: 20 Samsung S6: ~8 Optical sensor: 40 10 0 100 200 300 400 500 600 sensor size (mm 2 ) Optical sensor: 14.2mm 16mm Solid sensor1 : (13mm 13mm) Solid sensor2: (9.6mm 9.6mm) Samsung, S6: 10mm 4mm Apple : 4.5mm 4.5mm
소형지문센서의문제 Only the touched area produces a part of the whole fingerprint image. nail finger Overlapped region must be large enough. Nail to nail, a whole rolled fingerprint image
Types of Errors Performance Evaluation FAR(False Accept Rate), FRR(False Reject Rate) EER(Equal Error Rate): when FAR = FRR FTA(Fail to Acquistion, Input Reject Rate), FTE(Fail to Enroll, Decision Reject Rate) imposter Percentage(%) Decision reject range threshold genuine False reject False accept 0 Similarity: Normalized matching score 100
Fingerprint Verification Competition* (*https://biolab.csr.unibo.it/fvcongoing/ui/form/home.aspx) FV_STD-1.0
Accuracy Changes vs Sensor Size 192 Original Image 192 8 pixel Reduced Image Sensor size in Pixels Sensor size in mm Accuracy in EER (%) 192x192 (100.0%) 176x176 (84.0%) Sensor Size 160x160 (69.4%) 144x144 (56.3%) 9.6 x 9.6 = 92 8.8 x 8.8 = 77 8.0 x 8.0 = 64 7.2 x 7.2 = 51 1.75% 3.91% 8.69% 15.34%
Researches for Small Sensor Features other than minutia <edge shape of friction ridges> Minutiae + Ridge Flows (2014) Pores in a high 1000 resolution image <types of proposed micro ridge feature> Micro-features: BERC
Performance of BERC Methods 100 % FVC 2002 DB 1 90% 70% 50% 40% 30% Image Size (W x H) pixel 231.9 x 324.2 193.7 x 286.0 157.0 x 249.4 136.8 x 229.1 114.1 x 206.4 Sensor Size (W x H) mm 11.8 x 16.5 9.8 x 14.5 8.0 x 12.7 7.0 x 11.6 5.8 x 10.5 Minutiae-based 0.00 0.07 3.01 5.94 11.99 HoG-based 1.82 2.00 2.95 3.97 15.07 Proposed method 0.00 0.03 0.98 2.97 5.99
Smart Enrollment Use of partial and fused fingerprint images Fusion of fingerprint images By rubbing [ 관련특허 ] -KR 10-0613697 ( 등록 ) -PCT/KR2004/001794( 만료 )
등록영상 vs 지문인식성능 지문영역크기별, 등록영상별성능분석 192 Original Image 192 8 pixel EER (%) 지문면적 (mm) 7.2 x 7.2 (56.3%) 8.0 x 8.0 (69.4%) 8.8 x 8.8 (84.0%) 9.6 x 9.6 (100.0%) 5 장등록 (DB1) 18.59% 12.17% 7.04% 4.48% 10 장등록 (DB2) 15.34% 8.69% 3.91% 1.75%
ECG+ 지문인식성능 병행적융합결과 융합전심전도인식성능은 EER 3.5% (5 초데이터 ) 등록영상수 5 장 (DB 1) 10 장 (DB 2) 융합전지문인식 EER (%) 4.48 % 1.79 % 융합후 EER (%) 0.98 % 0.66 % + 위조지문검출효과있음
Mobile Touchless Fingerprint Recognition Using Built-in Camera www.yonsei.ac.kr
Depth of Field of the mobile camera is crucial for clear fingerprint image!
Recent Examples <Galuxy S5> # of minutiae: 46 <LG G Pro 2> # of minutiae: 43 <I-phone 5S> # of minutiae: 25 Samsung Galuxy S5 LG G Pro 2 Apple I-phone 5S Resolution 16 M (5312 x 2988) 13 M (4160 x 3120) 8M (2448 x 3264) Depth of Field In the macro mode (Easiness of image capture) Very good Very good Not so good
BERC: Window Guide Hand-shape guide for three fingerprints Easy/fast detection and segmentation for foreground finger image www.yonsei.ac.kr
Image Capturing for Touchless Fingerprint Recognition
Line Profile Checks for ROI Segmentation To check first finger is in the guide To check all three fingers are in the guide L Fingerprint segmentation <second-finger> Fitting check for input finger images <first-finger> www.yonsei.ac.kr
Performance example* *( 2013. 12. 1) Guide window (left fingers) Guide window (right fingers) Indoor condition, 5 image enrollment, S3/4 with 2 M pixel auto-selection (fusion of first and second fingerprints) FAR 10% 1% 0.7%(EER) 0.1% 0.01% GAR (FRR) 99.78% (0.22%) 99.35% (0.65%) 99.3% (0.7%) 98.9% (1.1%) 98.4% (1.6%)
Accuracy Change with Multiple Matches #of Matches 1:N 1:1 1:2 1:3 1:4 1:5 EER 1.1% 0.6% 0.4% 0.24% 0.19% Decide Yes for at least one Yes among N matches. www.yonsei.ac.kr
3. 모바일홍채인식 Pupil Iris Sclera - Iris pattern is different for different person. -Use a Near Infra-Red light (720-900nm) to illuminate the iris for getting a clear image
Sharbat Gula
홍채영상과근적외선조명 By Normal Mobile Phone Camera Phone Camera with flash-on With NIR (750~850 nm) Illuminator
Conventional Iris Recognition Oki IRISPASS-M 보편적홍채인식 : 줌및회전기능의카메라와양쪽조명사용
초기모바일홍채인식 Mobile iris scanner http://www.xvista.co.uk
PIER series Mobile Iris Recognition PIER (Portable Iris Enrollment and Recognition) handheld camera from Securimetrics, specializing in military and police deployments. http://www.securimetrics.com/ Operating range : 4 ~ 6, operating time : 15 frame/sec Dimensions : 8.9(W) 15.3(H) 4.6(D)cm 3 weight : 0.468 Kg Max. # of users : 200,000~400,000 subjects System speed : 1.33 MHz, X86 Display : 240 by 320 LCD touch screen
Mobile-Eyes TM www.retica.com operating time : 30fps for each eye accuracy : 99.96% TAR at 10-4 (fusion) Dimensions : 17.5(W) 20.5(H) 7.1(D)cm 3 weight : 1.134 Kg Interface: USB 2.0
Mobile Iris Rec. for Phone Unlocking Easy/fast user interface is the first choice for phone unlocking OKI mobile iris scanner: 2007 Basic feature: Generate/Compare iris data, Encrypt iris data Processing time: Authenticate in less than 0.5 seconds after capture Authentication accuracy: FAR<1/100,000 (Tested on a 2Mpixel mobile phone camera)
초소형홍채인식기 <http://www.i-lockglobal.com/>
eyed Biometric Password Manager By Winkpass Creations, Inc. Rear Mobile Camera with flash-on
Our Mobile Iris Recognition Extra NIR LEDs Extra Iris Camera 1 2 3 *Small space for NIR LED and Iris Camera *Issues for good iris image capturing *Locations for NIR LEDs (750~850 nm) and iris camera *Location for guide window showing user s iris image *Operating distance between phone and user s eye
Guide Window The system captures a good iris image automatically among the input image stream in real time. The window guide shows the incoming input eye images in real time. The window guide has an eye shape where the user fits his eye on it.
Location of Window Guide Iris Camera & LEDs are placed at the top O Guide at upper part. X Shade and occlusion by eyelid and eyebrow 63
Positions for Iris Camera, LEDs To avoid Red-eye effect or glint on glass, Camera and LEDs should be separated more than 5 degrees. Too far away makes a shadow at one side of eye. NIR LEDs Iris Camera 1 2 3 Sample 013 오른쪽눈화면상단1/4 중앙볼때 LEDs too close to camera LEDs far from camera Shadow on eye
Wavelength and Power of LEDs (a) four 750nm LEDs, good for iris boundary detection but dark image (b) two 750nm LEDs and one 850nm LED, still dark (c) two 850nm LEDs, good for bright iris image but less clear iris boundary
Operating distance Hot spot 20-25 cm Operating Distance (cm) Iris size in pixel 15 236 16 225 17 210 18 197 19 188 20 179 21 167 22 161 23 154 24 147 25 143 Choice for Hot Spot: -User convenience to see the window guide, -LED power -Iris image size -Camera focus 66
Performance Example* (*2013, BERC & Samsung) GAR = 100- False Reject Ratio = True Accept Ratio Enrollment Valid code size Recognition Valid code size > 1150 > 850 EER (%) 0.5105 FAR vs GAR (%) 0.0427 : 98.5078 0.1399 : 98.9440 ~0 : < 97.0 FTA Rate (%) 1.4 FTE Rate (%) 2.1 wearing no glasses False Accept Ratio (%)
Why is it still not appeared in the market?
Why is it still not appeared in the market? To use it so many times whenever unlock the phone, it is still not convenient enough. However, for phone-payment, it is adoptable. May appeared in a larger size Phone or Tablet PC, first.
Iris Rec. in a wearable (future appear?) *http://www.fidelyswatch.com/#!about/cjg9
4. 모바일장문인식 3 개의주손금 (Principal lines), 잔손금 (wrinkles), 아주작은잔주름은제거 71
Mobile Touchless Palmprint recognition* (* J.S. Kim et al, An Empirical Study of Palmprint Recognition for Mobile Phones, IEEE CE, August 2015.) Image Capturing with a Hand-shaped Guide
Image Capturing for Palmprint Recognition
Performance (*J. Kim et al, An Empirical Study of Palmprint Recognition for Mobile Phones, IEEE CE, Aug. 2015) Verification performance (in EER) DATABASE COMPCODE OLOF BOCV FCM PROPOSED MET HOD PolyU DB 0.09% 0.13% 0.15% 0.09% 0.11% BERC DB1 6.14% 5.14% 6.35% 5.48% 2.88% BERC DB2 5.87% 5.33% 7.64% 7.10% 3.15% IITD DB 6.33% 5.26% 5.69% 5.67% 5.19%
Performance by N Matches (*2013. 11. 15, BERC DB1) EER One time match 2.88% Five time matches 0.97% Performance Improvement by Multiple Matches www.yonsei.ac.kr
6. 모바일얼굴인식 Automatic Identification of a person from a face image captured by a camera. http://www.korea-id.co.kr/
Face Recognition by Statistical Features Statistical Methods : PCA, LDA, ICA, LMNF, KPCA, D-LDA, NMF, K-ICA P M i 1 p i u i G M i 1 g i u i
Different Persons?
Similar Faces: Owners and Dogs
모바일얼굴인식 Android Market - FaceLOCK by SmartApps Mobile Android Market - AppLock by Visidon 이후안드로이드 4.0 이상버전을 OS 로쓰는다수의스마트폰들이얼굴인식을통해잠금해제가능 [2] ( 예 : 삼성갤럭시시리즈, LG 옵티머스시리즈, 베가레이서등 ) Apple AppStore - FaceLock by App Impulse Cydia(Black Market) - RecognizeMe by Appcollipse
위조얼굴에의한공격 A photo face is used to log-in for Y430's Lenovo Veriface III authentication software* [*http://news.cnet.com/8301-17938_105-10110987-1.html]
위조공격의취약 문제점 : 보안취약 다른스마트폰에띄운사진으로도잠금해제가능 비슷한얼굴을가진사람또한인식된다고함 < 실제얼굴이아니라도 unlock 가능 > 해결 : blink detection 삼성에서는 blink detection을사용하여사진을통한보안해제예방 안드로이드젤리빈 (Jelly Bean) 버전에서부터 liveness check 기능이추가됨 < 새로추가된 liveness check 기능 >
BERC: Fake Face Detection Type of Fake Faces Print or Photo 3D Mask Video 2D Static Face Image 3D Imitated Facial Mask 2D Facial Image Sequence Replay
BERC Approach*: Use of all Frequency bands *G. Kim, Jaihie Kim, Face Liveness Detection Based on Texture and Frequency Analyse, ICB 2012. H 1W 1 1 Fuv (, ) f( xye, ) HW x 0 y 0 ux vy j2 ( ) H W 2D FFT 32 concentric ring bands Results of face liveness detection with three database Li s method Tan s method 1) From whole face From subblock NUAA 1) 76.7 % 86.7 % 92.4 % 83.4 % BERC Photograph 59.2 % 87.9 % 87.0 % 92.4 % BERC Print 68.9 % 88.4 % 89.8 % 92.6 % Feature dimension 1 4096 32 288 www.yonsei.ac.kr 1) X. Tan, Y Li, J. Liu and L. Jiang, Face Liveness Detection from a Single Image with Sparse Low Rank bilinear Discriminative Model, In Proceedings of European Conference on Computer Vision: Part VI, 2010.
Mobile 얼굴인식연구 issues 얼굴및카메라움직임에의한 blurring 보상 다양한환경변화 ( 조명, 배경 ) 에취약 얼굴인식의근본적성능미흡 보안에취약
7. 생체정보의 보호 Woman fools Japan's airport security fingerprint system reported by AFP 2009,1,29 A woman passed through Japanese immigration screening system by using tape on her fingers to fool a fingerprint reading machine. Fingerprinting at Japanese airport Fake Fingerprint
Fake Fingerprint Protection 조달청 전자입찰시스템
Fake Fingerprints <Paper> <Rubber> <Silicone> <Gelatin> < OHP > <Prosthetic finger>
LED for Fake Detection http://www.unioncomm.co.kr/ www.nitgen.com true www.dermalog.de fake
Fake Detection by Deformation Live Finger Gummy Finger A method based on the analysis of elastic characteristic of finger skin showed EER 4.9 % (Maltony, ICBA 2006)
Intrinsic Sensor Characteristics Some solid state sensors have the intrinsic characteristic sensing only live skins Among capacitive sensors, thermal sensors, E-field sensors Outer dead skin Live skin [Capacitive]
Spoof attack on S5* http://biz.chosun.com/site/data/html_dir/2014/04/16/ 2014041601068.html?news_Head2_01
300 250 200 150 100 50 0 1 11 21 31 41 51 61 71 81 91 101 111 121 Ridge signal Fake Detection: BERC* No feature offers complete fake detection. BERC approaches*: software/multiple feature classifier BERC Fake Detector Power spectrum Input Image Contrast Ridge thickness Ridge signal Gray-value Ridge signal Classification using Polynomial SVM Decision Live/spoof Ridge First order histogram 93
Experimental results* *Fingerprint liveness detection Using SVM-based Ensemble Classifier, W. Lee, H. Choi, J. Kim, ICEIC 2015 Real Finger FRR (False Reject Rate) 133/2000 (0.07) SDR (Spoof Detect) 1889/2000 (0.94) Paper SAR (Spoof Accept) 111/2000 (0.06) Fake Finger SDR (Detect) 2000/2000 (1.0) OHP Film SAR (Accept Error) 0/2000 (0.0) SDR (Detect) 829/1000 (0.83) Fake Finger Rubber Silicon Gelatin SAR (Error) 171/1000 (0.17) SDR (Detect) 1964/2000 (0.98) SAR (Error) 36/2000 (0.02) SDR (Detect) 1945/2000 (0.97) SAR (Error) 55/2000 (0.03)
Cancelable Biometrics Intentional distortion (transformation) of a biometric signal or feature to produce a different one. The transformation must be non-invertible. When stolen, a different transformation or same transformation with different parameters is applied. => changeable (cancelable) x Stored in the system Original x
Cancelable Fingerprints* x Original minitiae Move by a transformation Transformed minitiae *Chulhan Lee, Jeung-Yoon Choi, Kar-Ann Toh, Sangyoun Lee, and Jaihie Kim, Alignment-Free Cancelable Fingerprint Templates Based on Local Minutiae Information, IEEE Trans. on SMC (B), pp 980-992, Aug. 2007.
Iris recogntion Implemented in various standing distances; normal, far, portable High accuracy, but some optical issues on mobile phone Fingerprint recognition Small size, economical; most popular Touchless could be another solution for mobile applications Mobile biometrics Phone unlocking requires absolute user convenience; the manufacturer More mobile applications are explored requiring higher reliability and lower price rather than user convenience; phone-payment Biometric security Biometric data can be more secure than non-biometric data by technical privacy protection; spoof detection, cancellable biometrics/bio-hashing, watermarking Future 7. 맺는말 Multimodal and/or Multiple Matches at one time Biometrics in wearable devices www.yonsei.ac.kr
cherup.yonsei.ac.kr