Weblog Entries
100 NZ govt sites & Google
Friday, August 29, 2008
1. Introduction
For some .govt.nz websites, compliance with the NZ Government Web Standards and Recommendations v1.0[1] is mandatory since 1 January, 2008.
The NZ Web Standards are based on the international World Wide Web (W3C) Web Accessibility Initiatives (WAI).[2]
This survey is not intended as an accessibility audit, it was hoped to serve as a highlight of what is not accessible and aims to help identify areas for ongoing improvement in web standards authoring. There is no substitute for a full, manual website accessibility evaluation especially when it comes to involving users with disabilities and assitive technologies.
2. Methodology
Open Standards
The websites were selected for the survey using Google and were tested using the Open Accessibility Checks (OAC) project[3] to determine their compliance with international accessibility guidelines. The tests were performed on the most basic WAI standard, WCAG 1.0 (Level A)[4] as the baseline and a fundamental requirement for accessibility.
Level A conformance is a Priority 1 checkpoint that websites must satisfy, failing this basic requirement means that one or more groups will find it impossible to access information.
Website and Page Selection
We tested 100 of the most popular .govt.nz websites by using the Google search engine[5] page rank algorithm. The list of websites returned in the search suggested rich content and comprehensive navigation on the home page; these pages were selected for this survey test.
Automation
The tests were automated using Google and open source OAC software from the Adaptive Technology Resource Center (ATRC) at the University Of Toronto[6] and various tools from our garden shed.
3. Survey Results
Summary
Of the websites tested for Level A compliancy, 1 website was given an unconditional pass and 25 were given a conditional pass flagging potential accessibility problems, 72 failed on known problems and 2 websites were unavailable to test. (Figure 1).
Figure 1: WCAG 1.0 Priority 1 (Level A) Results
30 of the websites had known images problems and 61 had known script problems along with frames, links, style sheets and objects errors. (Figure 2)
Figure 2: HTML Group Results
The 1 website that scored a perfect pass (http://www.lawcom.govt.nz/) was in fact unable to be tested correctly by the checking software due to parsing incompatibilities and returned invalid code for the entire page that was used for the test: <HTML xmlns:fo="http://www.w3.org/1999/XSL/Format">
<HEAD></HEAD>
</HTML>
Number 2 in the accessibility ranking (http://timeframes.natlib.govt.nz) was using the meta refresh tag to redirect traffic to the website home page, it was the redirect page that was checked and not the true home page. (actual homepage link below) http://timeframes.natlib.govt.nz/logicrouter/servlet/LogicRouterOUTPUTXSL=home.xsl&hier=h1&tree=c&api_1=PUB_DISP_COLL&hier=builder&tree=c&api_2=GET_SEARCH_PARM&hier=h1&tree=o&api_3=PUB_DISP_COLL&hier=builder&tree=o&ds_svAPI_searchparm=4&api_4=GET_SEARCH_PARM&ds_svAPI_sortoptions=5&api_5=GET_USER_SORT_OPTIONS
The results as shown by HTML Group:
Images: 30 of the websites had known image problems and of the remaining 68 websites 25 had likely issues with images.
Scripts: 61 websites had known script problems and a further 21 had likely problems.
Links: 3 websites had known problems with links.
CSS: 57 websites had likely problems with style sheets.
Frames: 2 websites were still using frames.
Objects: 3 websites had known errors with objects.
4. Findability
We compared the Accessible ranking with the Google ranking. The Google results were re-ranked based on the highest accessible test score. The results were promising for an accessible search solution however, the top 2 websites listed by accessibility were passed yet the results were incorrect due to poorly formed code or a redirected homepage.
Popular does not mean accessible
Table 1 shows the top ten Google ranked websites, 7 had known failures that mean difficulty for some users accessing the website.
| Accessible Rank |
Google Rank |
Website | Result |
|---|---|---|---|
| 16 | 1 | http://newzealand.govt.nz/ | C PASS |
| 23 | 2 | http://www.ird.govt.nz/ | C PASS |
| 40 | 3 | http://www.workandincome.govt.nz/ | FAIL |
| 59 | 4 | http://www.immigration.govt.nz/ | FAIL |
| 52 | 5 | http://www.immigration.govt.nz/migrant/ | FAIL |
| 60 | 6 | http://www.dol.govt.nz/ | FAIL |
| 38 | 7 | http://www.jobs.govt.nz/ | FAIL |
| 56 | 8 | http://www.moh.govt.nz/ | FAIL |
| 4 | 9 | http://www.legislation.govt.nz/ | C PASS |
| 62 | 10 | http://www.minedu.govt.nz/ | FAIL |
Accessible should always be popular
Table 2 shows the top 10 websites ranked by standards compliance according to the automated accessibility checker.
| Accessible Rank |
Google Rank |
Website | Result |
|---|---|---|---|
| 1 | 71 | http://www.lawcom.govt.nz/ | A PASS |
| 2 | 90 | http://timeframes.natlib.govt.nz/ | C PASS |
| 3 | 91 | http://www.corrections.govt.nz/ | C PASS |
| 4 | 9 | http://www.legislation.govt.nz/ | C PASS |
| 5 | 94 | http://www.caa.govt.nz/ | C PASS |
| 6 | 24 | http://executive.govt.nz/ | C PASS |
| 7 | 60 | http://www.transit.govt.nz/ | C PASS |
| 8 | 15 | http://www.ltsa.govt.nz/ | C PASS |
| 9 | 53 | http://www.climatechange.govt.nz/ | C PASS |
| 10 | 38 | http://www.medsafe.govt.nz/ | C PASS |
5. Conclusion
Despite the number of websites with known problems, failing mainly on script issues and missing or incorrect image attributes, the survey produced a promising overall result due to the grouped nature of the errors.
Script errors were by far the most common issues flagged in checks. With automated tests the results cannot detect the severity of the error, so for example, low impact errors like a web stats counter missing the </noscript> element (see WCAG Samurai errata WCAG 1.0) or a potentially catastrophic loss of javascript navigation. For the websites we surveyed, low impact is assumed and evident.
On manual review, many of the websites were showing a professional and robust use of current authoring specifications.
As the survey was only the most basic Priority 1 (Level A) test using automated software, it should not be assumed that the public sector is barrier free.
6. Recommendations
The Priority 1 results should at least indicate that there is always an ongoing need to keep on top of accessibility issues. As the results should suggest, education and quality control for websites with hands on stake holders, content managers and contributors is especially important for images, linking and overall readability including people with English as a second language. If the website structure is robust, but the content is managed without due care, users may have problems.
Automated checking can be a helpful and informative way to quickly do a website health check. When looking for the absence of required elements and attributes these checks can be automated successfully. This is something that humans simply cannot do when involving large numbers of website pages.
Checking websites with automated software certainly has it's place in accessibility testing and website governance. Regular checks will assist with low level problem identification that could ultimately uncover larger issues that need to be addressed. Automated checking will save time, money and point out issues that could otherwise be missed. Regular testing needs to be undertaken as part of the ongoing process and commitment to ensuring websites are accessible. Automated software like people is not perfect, but in a world where we need accessibility, who is.
7. Bibliography
[1] New Zealand Government Web Standards and Recommendations v1.0
http://www.e.govt.nz/standards/web-guidelines/web-standards-v1.0/
[2] W3C Web Accessibility Initiative
http://www.w3.org/WAI/
[3] Open Accessibility Checks Project
http://www.atutor.ca/achecker/oac.php
[4] WAI Web Content Accessibility Guidelines 1.0
http://www.w3.org/TR/WAI-WEBCONTENT/#priorities
[5] Google
http://www.google.co.nz/
[6] Adaptive Technology Resource Centre, University of Toronto
http://atrc.utoronto.ca/
[7] W3C Validator
http://validator.w3.org/
8. About Us
Accessible.co.nz can help with periodic website health checking, we assist with compliancy on International and New Zealand Standards and guidelines. We also provide testing on assistive technologies through our local disability organisation.