[macvoiceover] Re: Programming with VO
- From: Bryan Smart <bryansmart@xxxxxxxxxxxxxx>
- To: "macvoiceover@xxxxxxxxxxxxx" <macvoiceover@xxxxxxxxxxxxx>
- Date: Wed, 1 Dec 2010 14:16:40 -0500
Tim, WX Widgets works fairly well with VoiceOver, but I'm pretty sure that they
don't support all of the native Mac user interface elements. When I started, my
priority was to get full control of creating Mac interfaces, not do something
that was cross-platform.
I've been told that there is a type of layout engine for WX Widgets that is
similar to the RenaissanceX idea. You might want to look at that. It might be
that WX Widgets are enough to make your program accessible on both Mac and
Windows.
If not, though, then you have the shell idea: abstract all of the main guts in
to a library, and then write a Mac and Windows shell application that makes
calls in to the common library. Ultimately, if you can stand it, that way is
best, as it lets you port your program to almost anywhere in the future.
While this thread is VoiceOver related, I think that we're going to soon
overwhelm the users with the techno-speak. Try to get on to mv-dev. If you
can't, then let me know.
Bryan
-----Original Message-----
From: macvoiceover-bounce@xxxxxxxxxxxxx
[mailto:macvoiceover-bounce@xxxxxxxxxxxxx] On Behalf Of Tim Burgess
Sent: Wednesday, December 01, 2010 3:49 AM
To: macvoiceover@xxxxxxxxxxxxx
Subject: [macvoiceover] Re: Programming with VO
Bryan,
I'll definitely be taking a look at using your stuff to port the PC project
that you've been kind enough to test for me. Currently, we're getting the
back-end code sorted out so that my initial win32 source gets replaced by Boost
and PortMIDI equivalents to ease the porting process. We're also making it
multi-threaded so that we can handle timers correctly and that's why you
haven't heard anything about the Reader for a while. Like your project, this
is almost certainly going to end up as freeware, so it's getting fitted in
around trying to feed the kids.
Have you looked at using WX Widgets for creating cross-platform UI elements?
Best wishes.
Tim Burgess
Raised Bar Ltd
Phone: +44 (0)1827 719822
Don't forget to vote for improved access to music and music technology at
http://www.raisedbar.net/petition.htm
-----Original Message-----
From: macvoiceover-bounce@xxxxxxxxxxxxx
[mailto:macvoiceover-bounce@xxxxxxxxxxxxx] On Behalf Of Bryan Smart
Sent: 01 December 2010 00:35
To: macvoiceover@xxxxxxxxxxxxx
Subject: [macvoiceover] Re: Programming with VO
Long delay in getting back to this thread, but...
Sorry that I misunderstood you. Thought you said that it wasn't possible at
all, not that it wasn't possible with Interface Builder.
The problem with Interface Builder isn't so much that it is inaccessible to
VoiceOver, as that it is an unsuitable type of program for a blind person to
use for this sort of work. Back when I started exploring programming options
for the Mac, I found lots of complaints about how IB didn't work. I discovered
that, with some workarounds, it actually did work, a bit. Most all of the IB
user interface is accessible. However, VoiceOver, as good of a screen reader as
it is, doesn't really provide much of a way for us to get a grasp on layout.
VoiceOver on iOS does a good job. You can feel around the screen, and know
exactly where controls are located. VO on OS X, with a track pad, does sort of
the same thing, but doesn't seem as accurate when it comes to layout. If you
interact with a toolbar, for example, you'll clearly see this demonstrated.
Moving horizontally on the track pad navigates between buttons, but the track
pad is completely oblivious to any sort of vertical spacing. Now, I know that a
tool bar is just one line, so it shouldn't really matter. In practical
day-to-day terms, the current way is probably the better way. Excluding this
example, this sort of zoning happens in other places. Desktop VO might split
the track pad in left and right halves to let you know there is a side bar
area, but the proportions that it chooses don't seem to be related to the
actual size on the screen. So, on iOS, the tactile position is very literal,
but, on the desktop, the spacing is meant more to convey meaning to you, rather
than to accurately represent the screen. Given this, it is very difficult to
imagine using a program like Interface Builder, regardless of its accessibility
level. Suppose you're in a window, containing a button and some static text.
Since VO isn't showing you literal positions, how can you drag the button to
the right location?
The answer is that you can't.
When it comes to formatting, IB includes some positional guides that help line
up controls. These are like grids that are overlaid on the window. If one of
these grids is enabled for a part of the window, then you're shown little
boxes, and you drag a button in to one of the boxes, then IB snaps that button
to an appropriate exact position. I have no idea how that sort of info could be
conveyed to a VoiceOver user. Maybe each zone of the grid/guide could be a
blank container that they'd interact with? Except, if you did, you'd need to
perform the release step of drag and drop there, and with an empty container,
I'm not sure if you could use VO-Command-F5 to move the mouse to the position
prior to releasing the item being drug.
After realizing all of this, I was a lot less upset with the lack of
accessibility, or, more accurately, the lack of my ability to accomplish
anything meaningful with IB. It wasn't a matter of Apple needing to add labels
for controls, or anything so straight forward. The program was fundamentally
designed to be a visual tool, and trying to make it work without vision is a
challenge that would require changes in not only it, but in the screen reader,
too. Quite a huge project, to be sure. That's a bit like asking Microsoft to
make Visio accessible.
Incidentally, in IB, you can set hierarchical relations between controls by
using cut/copy/paste in the outline view. You can open the inspector for any
graphical control, and set its position by manually entering coordinates.
You can establish data bindings by directly editing in xCode, and IB will pick
up on them. Still, while you "can" work this way, it requires a massive amount
of effort. The effort, in my mind, is too much. For most people, the user
interface is the easiest part of the app to slap together. For us, this really
is like trying to use a screw driver to hammer in a nail, since the whole
process is counterintuitive to the way that we think about user interfaces.
I don't know how much you understand about how IB works behind the scenes, but
any work that a person performs for interface design in IB is saved as NIB/XIB
markup files. When your app starts, Cocoa loads this file from the bundle, and,
based on the parameters in the markup file, creates the needed objects that
make up your user interface. This is exactly what happens with Renaissance,
except the parameters just originate in a differently formatted markup file.
Apple assumes that most people will be using Interface Builder, so their markup
is optimized for small size and speed of loading, rather than to be human
readable and editable. The results are the same, though.
Renaissance also includes the extra step of automatic formatting, if you want
it.
I hate teasing people with pre-announcements, but, when threads like this come
up, it is hard not to do so. The Renaissance markup is quite easy to read and
edit by hand. Still, there is no reason that it must be read or edited by hand.
Just like how Interface Builder is really an editor for NIB/XIB files, it is
certainly possible to make an editor for the GSMARKUP files used by
Renaissance. That sort of program is one of my current projects. I'm currently
calling it Interface Describer. *smile* Once finished, it can be registered as
the creator/owner for GSMARKUP files, and, when that happens, you'll be able to
simply open the GSMARKUP files in your xCode projects in the way that sighted
people open the NIB/XIB files. It will be pretty much a full replacement for
Interface Builder at that point for people that want to use dynamically
generated interfaces. Since the objects that Renaissance creates at run-time
support the interfaces of the Cocoa objects, someone using Interface Describer
to create their interface need know nothing about Renaissance to use it. They
edit their interface with Interface Describer, and code like they're using
regular Cocoa objects.
The transparent operation is a cool concept for me to contemplate.
The Java and Mono stuff might be useful, in ways, but, as far as I know,
Renaissance is the only way to do this sort of thing if you want to write
native Object C apps.
I'm glad to hear that you use it and like it. I was expecting way more
excitement and participation with it than has happened. That is really
disappointing. Since this is a free project, it is the enthusiasm of the people
using it, and seeing the software that they build with it, that would give me
the energy to keep working on it. I guess there aren't too many blind people
that are interested in programming for OS X. An iOS version might be better
received. With the desktop version, I was able to build on the massive work
that was already undertaken for the GNUstep version. On iOS, the work would
involve almost an entire rewrite. I have full-time work and other hobbies,
though, so don't know if I have that kind of time. Maybe if I could split the
work with a group of people, we could make it happen, but, so far, programming
interest among blind people for any Apple stuff has been low, as best as I can
tell.
If you're not on MV-Dev, think about coming over. I really wish that we could
get more traffic going on that list with blind coders.
Bryan
-----Original Message-----
From: macvoiceover-bounce@xxxxxxxxxxxxx
[mailto:macvoiceover-bounce@xxxxxxxxxxxxx] On Behalf Of Travis Siegel
Sent: Thursday, November 25, 2010 1:51 AM
To: macvoiceover@xxxxxxxxxxxxx
Subject: [macvoiceover] Re: Programming with VO
On Nov 24, 2010, at 7:32 PM, Bryan Smart wrote:
> Travis, this isn't correct. My RenaissanceX library will automatically
> generate user interfaces at run-time that conform to Apple's Human
> Interface Guidelines. If you wish, it will also properly format
> (arrange/size/space) controls, based on their logical relationships.
> You can override that automatic formatting behavior for a single
> control, a group of controls, or as much of the UI as you'd like.
Right, all of which is done without interface builder.
The main point of my email was that ibuilder is *not* accessible other than
modifying existing interfaces already built in interface builder.
I did mention there were other tools, though I didn't specifically mention
renaissance, though I do have it, and mono as well, along with sdl, and sfml,
all of which can be used to varying degrees to accomplish interface building
(some definitely better than others), but the mere fact that the tools provided
by apple are *not* usable by voiceover users is a real kick in the pants, since
they have put so much work into voiceover itself. It's really a shame if you
ask me, but since nobody did, I guess it doesn't make a whole lot of difference.
>
> Click on the link below to go to our homepage.
> http://www.icanworkthisthing.com
>
> Manage your subscription by using the web interface on the link below.
> //www.freelists.org/list/macvoiceover
>
> Users can subscribe to this list by sending email to
> macvoiceover-request@xxxxxxxxxxxxx
> with 'subscribe' in the Subject field OR by logging into the Web
> interface at //www.freelists.org/list/macvoiceover
>
>
> Click on the link below to go to our homepage.
> http://www.icanworkthisthing.com
>
> Manage your subscription by using the web interface on the link below.
> //www.freelists.org/list/macvoiceover
>
> Users can subscribe to this list by sending email to
> macvoiceover-request@xxxxxxxxxxxxx
> with 'subscribe' in the Subject field OR by logging into the Web
> interface at //www.freelists.org/list/macvoiceover
>
>
> Click on the link below to go to our homepage.
> http://www.icanworkthisthing.com
>
> Manage your subscription by using the web interface on the link below.
> //www.freelists.org/list/macvoiceover
>
> Users can subscribe to this list by sending email to
> macvoiceover-request@xxxxxxxxxxxxx
> with 'subscribe' in the Subject field OR by logging into the Web
> interface at //www.freelists.org/list/macvoiceover
>
>
> Click on the link below to go to our homepage.
> http://www.icanworkthisthing.com
>
> Manage your subscription by using the web interface on the link below.
> //www.freelists.org/list/macvoiceover
>
> Users can subscribe to this list by sending email to
> macvoiceover-request@xxxxxxxxxxxxx
> with 'subscribe' in the Subject field OR by logging into the Web
> interface at //www.freelists.org/list/macvoiceover
>
Other related posts:
