Standardization of Cleaning Robot Performance

IEEE IROS 2010
Workshop on Performance Evaluation and Benchmarking for
Intelligent Robots and System with Cognitive and Autonomy
Capabilities
Standardization of Cleaning Robot
P f
Performance
M
Measurementt iin IEC
2010. 10. 22
Sungsoo Rhim, Ph.D.
Convener
off IEC
C
C TC
C 59/SC
9/SC 59F/WG
9 / G5
Kyung Hee University
Department of Mechanical Engineering
ssrhim@khu.ac.kr
Contents
 Overview of IEC
– IEC
 History and Plan of IEC TC 59/SC 59F/WG 5
 Current Discussion
– Cleaning Robot Performance Measurement
 Remaining Issues
2
IEC Overview
 IEC (International Electrotechnical Commission)
– The IEC is the world's leading
g organization
g
that p
prepares
p
and
publishes International Standards for all electrical, electronic and
related technologies — collectively known as "electrotechnology".
– http://www.iec.ch/
http://www iec ch/
3
IEC Structure
4
IEC TC 59
 TC 59: Performance of household and similar electrical
pp
appliances
– Secretariat : Germany
– Scope :
T prepare International
To
I t
ti
l Standards
St d d on methods
th d off measurementt off
characteristics which are of importance to determine the
performance of household and similar electrical appliances and are
of interest to the consumer. This may include associated aspects
related to the use of the appliances and aspects such as the
classification, accessibility and usability of appliances, ergonomic
characteristics and conditions for the information provided at the
point of sale.
– Note - "similar"
similar doesn
doesn'tt mean appliances for industrial use
Note : TC 59 does not deal with appliances which are already
explicitly covered by the scope of other IEC or ISO TCs
TCs.
5
Subcommittees(SCs) in TC 59
 SC 59A - Electric dishwashers
 SC 59C - Heating appliances
 SC 59D - Home laundry appliances
 SC 59F - Floor treatment appliances
– Title change ?: Surface cleaning appliances
– Scope :
To prepare international standards on performance measurement
methods for floor treatment appliances
 SC 59K - Ovens and microwave ovens
ovens, cooking ranges and
similar appliances
 SC 59L - Small household appliances
 SC 59M - Performance of electrical household and similar
cooling and freezing appliances
6
SC 59F Member Countries Participating WG 5
 P-members (12)
– Brazil, China, Denmark, France, Germany, Italy, Japan,
K
Korea,
N th l d Sweden,
Netherlands,
S d
U it d Kingdom,
United
Ki d
U it d States
United
St t
of America
– Participation
p
((8+1))
 O-members (23)
– Australia, Austria, Belgium, …
7
Working Groups in SC 59F`
 WG 3 - Maintenance of IEC 60312 (Vacuum Cleaner)
 WG 4 - Methods of measuring the performance of wet
vacuum cleaners
 WG 5 - Methods of measuring
g the performance
p
of
household cleaning robots
– Convener :
P f
Professor
Sungsoo
S
Rhim
Rhi (KR)
– Scope :
To p
prepare
p
draft standard on measuring
g methods on p
performance of
household cleaning robots
– Title Change ?:
S f
Surface
Cleaning
Cl
i R
Robots
b t
 WG 6 - Commercial Vacuum Cleaners
8
Incubation of WG 5
 IEC Taskgroup in SC59F: 2004
 Korean Standard : April 2006KS B 6934
− KS B 6934, First of its kind
 IEC TC 59 Meeting in Korea: October 2006
− Presentation
P
t ti off KS B 6934 and
d Di
Discussion
i ffor NWI
 IEC Taskgroup Meeting in Frankfurt: September 2007
– Presentation of KS B 6934 and Discussion for NWI
 IEC TC 59/SC 59F Meeting in Sydney: September 2008
– Discussion of NWI for WG5
 Proposal NWI 59F/183: November 2008
 Voting Results and Comments
 Announcement of RVN59f-183: Mar, 2009
– WG 5 is established
9
History of WG 5
 Meeting #1 : May, 2009, Seoul
– Sponsored
p
by
y LG, Samsung
g
 Meeting #2 : Sept, 2009, Stockholm
– Sponsored by Electrolux
 Meeting #3 : Feb, 2010, Boston
– Sponsored by iRobot
Submission of CD: End of March
Circulation of CD (2 mo)
 Meeting #4 : July
July, 2010
2010, Seoul
− Sponsored by LG, Samsung, Sejong Univ, Yujin
 Meeting #5 : Oct
Oct, 2010
2010, Seattle
−
74th IEC General Meeting
10
Standard Development Procedure
Project stage
Normal
procedure
Draft
Submitted with
proposal
Proposal stage
(see 2.3)
Acceptance of
Proposal
Acceptance
of Proposal
Preparatory
stage
(see 2.4)
Preparation
of working
draft
Committee
stage
(see 2.5)
“Fast-track
Procedure” 1 )
Acceptance
of Proposal
Technical
Specification 2 )
Technical
report 3 )
Publicly
Available
Specification 4 )
Acceptance
of Proposal
Acceptance
of Proposal
Study by
working
group
Preparation
of draft
Approval of
draft PAS
Development
and acceptance
p
of committee
draft
Development
and acceptance
of committee
draft
Acceptance
p
of draft
Acceptance
p
of draft
Enquiry stage
((see 2.6))
Development
and
acceptance of
enquiry draft
Development
and
acceptance of
enquiry draft
Acceptance of
enquiry draft
Approval stage
(see 2.7)
Approval of
FDIS
Approval of
FDIS
Approval of
FDIS
Publication
stage
(see 2.8)
Publication
of
International
Standard
Publication
of
International
Standard
Publication
of
International
Standard
Publication of
Technical
specification
Publication
of Technical
Report
11
Publication
of PAS
Project Stages
Associated document
N
Name
Abb
Abbreviation
i i
Preliminary stage
Preliminary work item
PWI
Proposal stage
New work item proposal
NWP
Preparatory stage
Working draft(s)
WD
Committee stage
Committee draft(s)
CD
Enquiry stage
Enquiry draft
ISO/DIS
IEC/CDV
Vote
Approval stage
Final draft international standard
FDIS
Vote
Publication stage
International standard
ISO, IEC or
ISO
ISO/IEC
12
N
Note
Plan
 Meeting #5 : Oct 2010, Seattle, USA
Ring test
 Meeting #6 : Feb 2011, Europe (Vorwerk, Germany)
Submission of CD #2: Mar 2011
Circulation of CD #2 (2 mo)
 Meeting #7 : June 2011, China (Tek, Shoju)
Preparation of CDV (2mo,
(2mo Bilingual Draft may be optional)
Submission of CDV: July, 2011
Circulation of CDV ((5 mo))
 Meeting #8 : Jan 2012, USA?
 Meeting
g #9 : May
y 2012,, Europe?
p
Submission of FDIS: May, 2012 (2 mo + 4 wk )
Publication of IS: Aug, 2012(2 mo + 4 wk )
 Two meetings/year for new issues and revision
13
Approval of Drafts
 An enquiry draft (CDV) is approved if
a) a two-thirds majority of the votes cast by the P-members of the
technical committee or subcommittee are in favor, and
b) not more than one-quarter of the total number of votes cast are
negative.
Abstentions are excluded when the votes are counted, as well as
negative votes not accompanied by technical reasons.
 A final draft International Standard (FDIS) having been
circulated for voting is approved if
a)) a two-thirds
t
thi d majority
j it off the
th votes
t castt by
b the
th P-members
P
b
off the
th
technical committee or subcommittee are in favor, and
b)) not more than one-quarter
q
of the total number of votes cast are
negative.
Abstentions are excluded when the votes are counted, as well as
negative votes not accompanied by technical reasons
reasons.
14
Participating Companies and Organizations














Dynson
Electrolux
Hanwool
HHI
iR b t
iRobot
Karcher
LG
Mearsdon
Philips
Samsung
Tek
Vorwerk
Yujin
and
a
d More
oe
15
Task for WG 5
 Performance measurement of Household Cleaning Robot
 What is the better cleaning robot?
 What do we mean by “Better”?
– C
Cleaning
ea g well
e
 What do we mean by that really?
–
–
–
–
–
–
More dust: Vacuum suction, side brush, air blower
More coverage: Autonomy, Navigation, Intelligence
Faster
Any environment?
consumption product cost,
cost maintenance,
maintenance life cycle
Cost: Power consumption,
cycle,
Noise
16
Performance Criteria
 Dust Removal (Collection) Performance
– Wooden floor, Carpet
p floor
 Autonomous Navigation Performance
– measure the coverage rate in a standard test environment with
various
i
obstacles
b t l
 Operation-time per Full-charge
– Source of major complain from the user (drastic decrement of
operation-time per charge)
 Fiber Pick-up
p
 Corner/Edge Dust Pick-up
 Fall-off
 Docking
 Noise (removed)
 Others
17
Navigation (Coverage) Test
Initial Version (Current KS)
– 5 m x 5m
– No dust
– Hardwood
– Sofa
– Table
– Chair
– Drawer
– Drop zone
– Lamp
– Threshold
18
Navigation (Coverage) Test
19
Navigation (Coverage) Test
Current
5mx4m
No
Dust
More
Obstacle
20
Navigation (Coverage) Test
500
2000
2
500
Pa
rtit
io
ite ning
m
4 Starting
Positions
21
Obstacles in Navigation Test
Item
Q’ty
Dimensions* (mm)
Weight
Colour
Remarks
Dresser
1
1000 (L) x 500 (W)
Heavy enough so that it d White
oes not move by robot, or
fixing the legs on the floor
Table
1
1000 (L) x 600 (W)
Fixed on the floor
Natural Cherry 4 legs of 40 x 40
Chairs
4
450 (L) x 450 (W)
Fixed on the floor
Natural Cherry 4 legs of 40 x 40
Sofa
1
2000 (L) x 600 (W)
Same as in Dresser
White
Sofa drop skirt
1
2000 (L) X 120 (H)
Fixed on the top of frame
Beige
Leg height of 125 mm, size is 48 mm in di
a.
Only in front of sofa. Made with polyester.
Partitioning Item
1
1000 (L) x 500 (W)
Same as in Dresser
18% grey
No legs, all sides are enclosed
Floor lamp
1
Diameter 330
Same as in Dresser
Metal or white With slope of 10 degree around
Floor electrical
wire
1
6 mm in dia.
500 mm Long
es o d (round)
( ou d)
Threshold
1
15
5 ((D)) x 500 ((L))
One end fixed with lamp b Black
ase, one end fixed on the
wall
N/A
/
Not
ot red
ed
Heater baseboard
1
Area rug with tassels
on the short edge
1
2000 (L) x 40 (W) x
Secured on the wall
250 (H)
1800 (L) x 1200 (W) X
10
Natural
Cherry
Ivory
Checker board
1
Black and
White
Metal transition
1
1000 (L) x 1000 (W) X
7 (H)
36 (W) x 2 (T)
36.5 (W) x 11 (T)
Wooden transition
Recess door
1
Aluminium
u
u
Wilton type area rug with tassels.
Tassels on both 1200 sides. Tassel size is
60 mm long,
g, with 4mm diameter,, and den
sity of 2 tassel/20 mm. Pile height is 7.0 m
m, carpet is 10 mm thick.
Not larger than 150x150X7, surface finish
shall be polished
Aluminium (See figure 8 for installation)
M-D Building Products
36"L x 2"W.
Model #43858, polished
Wood (see figure 9 for installation)
B
N t
lR
d
M
d l # 1117781
Bruce
Natural
Reducer.
Model
0, finished wood
900 (L) x 300 (H)
22
Navigation (Coverage) Test
 KITECH (Korea Institute of Industrial Technology)
A
B
C
D
E
F
G
H
I
Cleaning Head
49%
70%
61%
64%
59%
54%
67%
66%
61%
Cleaning Head +
Side-brush
92%
92%
91%
64%
75%
75%
67%
86%
92%
Whole Size of Robot
(on a maximum
length of robot basis)
94%
93%
95%
88%
84%
79%
90%
89%
95%
[Cleaning Head]
[Cleaning Head & Side Brush]
23
[Whole Size of Robot]
Test Dust
 Hardwood test : 80% of ISO 679 + 20% of IEC-Sand for
Hardwood
 Carpet test: 80% of ISO 679 + 20% of IEC-Dust for
Carpet
 Open to other suggestion based on the ring test results.
Particle size range
[mm]
Parts by weight
[%]
< 0
0,020
020
0,020
< 0,040
0,040
< 0,075
0 075
0,075
< 0
0,125
125
0,125
< 0,25
0,25
< 0,5
05
10
0,5
< 1,0
1,0
< 2,0
20
10
10
10
20
16
11
3
IEC 60312 5.1.2.1
Dolomite Sand
ISO 679
Sand
Mixture
(20%IEC Sand+80%ISO Sand)
24
Test Dust
 KITECH (Korea Institute of Industrial Technology)
A
B
C
D
E
IEC 60312 5.1.2.1
g)
Dolomit Sand ((200g)
30%
4%
49%
39%
60%
ISO 679
Sand (100g)
72%
26.2%
12%
51%
78%
59%
14%
21%
70%
66%
Mixture (100g)
(20%IEC Sand+80%ISO Sand)
121
IEC 60312 5
5.1.2.1
ISO 679 Sand for Cement
25
Non-Laminated
Non
Laminated Pine Wood
600
10
W
Dust Collection – Box Test VS Straight Line Test
26
Standard Carpet
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
type
pile composition
method off manufacturing
f
color
backing
g
type
total height
pile Height
total Weight/m²
pile weight/m²
number of Knots/m²
reed
shots
standard width
tolerances
Wilton
wool 8,6/2*2
Wilton fabric
f
dark, one color
Jute and Cotton + latex
cut -pile
7,5 mm
see also tolerances
6 4 mm
6,4
see also tolerances
2100 g/m²
see also tolerances
1500 g/m²
see also tolerances
96 000 knots/m² see also tolerances
320 r/m
300 sh/m
400 cm
+/- 5 %
27
Coner/Edge
Coner
/Edge Cleaning Test
 Dust pick-up near
g and corner
edge
 Dust distribution
on blue area
28
Fiber Pickup Test
 Hair (human, pet)
 Fiber
 Amount
29
Fall--off Test
Fall
 Being modified
30
Docking Test
 Different distance, Different angle
 Failure test
31
Energy Consumption
 EU Regulation on Standby Energy Consumption
32
Side Brush Impact
33
Side Brush Impact
34
Coverage Test
35
Corner/Edge Dust PickPick-up Test
36
Issues being Discussed
 Orthogonality
– Factoring
g independent
p
capabilities
p
VS p
practical realization
– Decoupling MOBILITY (NAVIGATION) from DUST PICK-UP
– Overall performance VS individual performance
 Fairness
– Normalization (Capability/Cost): Battery power, Motor spec, Size, …
– Features
F t
VS Level
L
l off technology
t h l
(quality
(
lit off particular
ti l ffunction)
ti )
• Vision based navigation VS Random navigation
• Side-brush and other accessories
 Weighting
– Which factor is important enough? How much?
 Comparison with existing similar product
– Cleaning Robot VS Vacuum Cleaner
 Consumer Perception
37
More Issues being Discussed
 Lighting Condition
– Intensity
y
– Temperature
 Visual Pattern
– Shape, Marks, Color
– Interference with tracking system
 Configuration
C fi
ti off obstacles
b t l
– Consistent results
– Level of difficulty
 Test Dust
 REPEATABILITY
38
Ring Test
 Test Method
–
–
–
–
Multiple samples (no shipping)
2 robot samples per model per place
14 robots
b t per mode
d are needed
d d
Samsung, iRobot, TEK agreed to provide 14 samples per model
 Test Group (blue for test leader)
– D
Dustt C
Collection
ll ti :Box
B T
Testt : iR
iRobot,
b t Samsung,
S
Phili
Philips,
D
Dyson, T
Tek,
k LG
LG, V
Vorwerk
k
– Dust Collection: Straight Line Test : iRobot, Samsung, Philips, Dyson, Tek , LG
, Vorwerk
– Navigation
Na igation : iRobot,
iRobot Samsung,
Sams ng Philips,
Philips D
Dyson,
son LG
– Corner/Edge : iRobot, Samsung, Philips, Dyson, Tek, LG, Vorwerk
– Fall-off : iRobot, Samsung, Philips, Dyson, Tek, LG, Vorwerk
– Fiber
Fib : iR
iRobot,
b t S
Samsung, Phili
Philips, Dyson,
D
T k LG,
Tek,
LG Vorwerk
V
k
– Docking : iRobot, Samsung, Philips, Dyson, Tek, LG, Vorwerk
– Operation time : iRobot, Samsung, Philips, Dyson, Tek, LG, Vorwerk
– Energy Efficiency
ff
: iRobot, Samsung,
S
Philips, Dyson, Tek , LG,
G Vorwerk
 Test Sample
–
iRobot 560 (random), Samsung NaviBot (vision), Tek D540 (random), LG VR5901 (vision)
 Time
–
Three month from now
39
Thank you !
Q&A
40
IEC Standards on Service Robots
Number
Title
Status
IEC 60312-3 Ed. 1.0
Methods of measuring the perfo Approved CD
rmance of household cleaning r in 2010
obots
IEC 60335-2-2 Ed6.0
Household and similar electrical Published,
appliances - Safety - Part 2-2: P 2009-12-14
q
for vacuu
articular requirements
m cleaners and water-suction cl
eaning appliances
60335 2 107 Ed.
Ed 1.0
10
IEC 60335-2-107
Household and similar electrical Approved
appliances - Safety - Part 2-107: CDV
Particular requirements for robot 2010-03-12
ic battery powered electrical la
wnmowers
41
Noteworthy Subjects in New IEC 6033560335-2-2
Clause
Mechanical strength
Construction
Contents
An evenly distributed load of 60 kg is placed on top of the mobile part for 60
sec. During this test, no short circuit shall occur. After the test, there shall be
no visible damage that could impair compliance with the standard.
Mobile parts of automatic battery-powered cleaners shall be equipped with
– a device to stop movement within 1 sec of accessible hazardous moving
parts when they lose contact with the surface being cleaned, and
– a device to protect the appliance from dropping off the cleaning surface
(e.g. stairways, etc.). When the mobile part senses that it has reached a
critical edge, it shall reverse and move away from the edge of the
cleaning surface and then continue to operate normally.
When operating on a sloped surface, the speed of the mobile part shall not
be excessive. For the test, the mobile part is directed to move down a glass
surface inclined at 10 deg to the horizontal and its speed is measured
measured.
The measured speed shall not exceed the speed measured on the flat
surface by more than 10 %.
42
Noteworthy Subjects in CD of IEC 6033560335-2-107
Clause
Sub clause
Sub-clause
Contents
Stability and
mechanical
ha ards
hazards
Controls
Operator presence control, Traction drive, Remote s
etting device, Manual controller, Emergency stop
Safety
Requirements
Cutting means enclosure, Cutting means stopping ti
me, Thrown object hazard, Access to cutting means,
Stopping distance, Noise
Mechanical
M
h i l
strength
G
General
l
Strength of cutting
c tting means and ccutting
tting means mo
mounti
nti
ngs, Imbalance, Structural integrity, Strength of cutti
ng means enclosure
Construction
Disabling device
Removable disabling device
device, Code protected disabli
ng device.
Working Area
Perimeter delimiter, Working area programming
S
Sensors
Tilt sensor, Ob
Obstruction
t ti sensors, Lift sensor, P
Proximi
i i
ty Sensor
Manual controller
Drop test
Batteries and
accumulators
Battery/Accumulator type, Terminal protection
Cutting means
start-up Warning
Warning
g light
g or audible indicator,, Time interval befo
re start-up
43