[REVIEW REQUEST] Improvements to TableView sorting

[Jonathan Giles]

Hi all,

I am wanting to summarise my plans on how to improve TableView sorting
in JavaFX 8.0, given the conclusions reached regarding the introduction
of SortedList in JavaFX 8.0 [1]. Apologies for the length of this email.
If I had more time I would have made it shorter.

OVERVIEW
====================
For a while I've maintained an umbrella issue for improved sorting
functionality in TableView (RT-19479 [2]). My expectation is what I
outline below should work towards resolving as many of the issues
outlined in this Jira issue, most importantly the following features are
ones I would love to support:

1) Support returning to an unsorted state.
2) Support forcing a 'continuous' sort (so that whenever the items list
is changed the TableView remains sorted).
3) Support custom sort policies (to allow for people to override the
current in-memory sort policy and instead, say, go direct to a database
to do the sort).
4) Be notified of when a sort event is about to occur and prevent it
from happening.
5) Make it possible to force a sort when something elsewhere in the
universe (unrelated to the tableview) has changed (e.g. when you know
the tableview is no longer consistent and should re-run its sort policy).

I'm not aware of other features that should be supported. If you have
some and they are within the scope of TableView sorting, then please
read the rest of this email and then reply to it so we can discuss.

IMPLEMENTATION
====================
I have attached an early draft patch for the following functionality
inside RT-19479. It is 'early and draft' in the sense that I'm fairly
comfortable with the API, but there is still a lot of implementation and
javadoc work that needs to be done. Hopefully it should give you all an
idea of what is planned, so that you may offer any thoughts before I get
too far into implementation and javadoc.

To summarise the patch, what I've done is add a new SortEvent class, and
then for both TableView and TreeTableView I have done the following:

* I've introduced a public static Callback DEFAULT_SORT_POLICY that
encapsulates the current sorting code. Users are free to overwrite
this sort policy should they wish to instead go to the database, for
example. They do this by calling tableView.setSortPolicy(...).
* I have made the TableView and TreeTableView sort() methods public,
but they simply defer to the sort policy (assuming it is non-null,
which is the case by default as the DEFAULT_SORT_POLICY is set by
default).
* I have made the code that calls sort() more full-featured. This
happens in a few places, most notably when the 'sortOrder' list is
changed (where sortOrder is a list of TableColumns which the user /
developer has specified are to be used to sort the data in the
table). In particular, I now do the following when the sortOrder
changes (or any of the other sort criteria relevant to the sort):
o I update the Comparator that I am about to use. This Comparator
is now publicly readable (but not writable), so that people can
do
sortedList.comparatorProperty().bind(tableView.comparatorProperty()).
+ The reason why I don't want the comparator to be writable is
that it would then allow for the comparator to be
inconsistent with the sortOrder list, whereas I view the
comparator as a reflection of the sortOrder. If people don't
want to use this comparator they are free to set their own
sort policy and use whatever they want in there.
o I fire the SortEvent. If this event is consumed I do not run the
sort().
o I call sort()
o sort() is now very simple: it calls getSortPolicy() and runs it,
passing in the TableView / TreeTableView and expecting a boolean
response to indicate success or failure.

The above outline essentially allows for features 2), 3), 4) and 5) from
the list above. The only feature missing therefore is 1), which is
support for returning to an unsorted state. The solution to this is to
have a special-case support inside TableView / SortedList such that the
developer may do what I mentioned above (namely,
sortedList.comparatorProperty().bind(tableView.comparatorProperty())),
and when the tableView.sortOrder list is empty the TableView /
SortedList will take that to mean there is no sort desired and to return
to the original, unsorted state.

The other important note to make is that if you as the developer provide
the TableView with a SortedList this is telling the TableView not to
sort the collection or modify the Comparator if the user / developer
changes the sortOrder (e.g. by clicking on the column headers). In other
words, a TableView with a SortedList can not have its sort order changed
by default. Of course, if you DO want the Comparator to be changed in
your SortedList (and to support unsorted state as above), you can simply
bind the comparator of the list to the tableview comparator.

I hope that is clear. It's a lot of detail to get out, but hopefully it
meets the expectations of everyone and I can go forth and implement it
(and pray for no gnarly edge cases). I think the most probable area for
issues is around maintaining consistent visuals for the sorted table
column(s) given the ability for sorting to fail and / or be ignored by
the developer implementation of the comparator / sort policy.

As always, your thoughts and comments are appreciated, but lets keep
this discussion focused solely on sorting in TableView (and to a lesser
degree TreeTableView).

[1] See the last comment by Richard Bair here:
http://javafx-jira.kenai.com/browse/RT-17053
[2] http://javafx-jira.kenai.com/browse/RT-19479

Thanks,
-- Jonathan

[Knut Arne Vedaa]

Post by Jonathan Giles
The above outline essentially allows for features 2), 3), 4) and 5) from
the list above. The only feature missing therefore is 1), which is
support for returning to an unsorted state. The solution to this is to
have a special-case support inside TableView / SortedList such that the
developer may do what I mentioned above (namely,
sortedList.comparatorProperty().bind(tableView.comparatorProperty())),
and when the tableView.sortOrder list is empty the TableView /
SortedList will take that to mean there is no sort desired and to return
to the original, unsorted state.
The other important note to make is that if you as the developer provide
the TableView with a SortedList this is telling the TableView not to
sort the collection or modify the Comparator if the user / developer
changes the sortOrder (e.g. by clicking on the column headers). In other
words, a TableView with a SortedList can not have its sort order changed
by default. Of course, if you DO want the Comparator to be changed in
your SortedList (and to support unsorted state as above), you can simply
bind the comparator of the list to the tableview comparator.

This is not very clear to me. It seems that you indicate there are two
scenarios:

A) The list you provide to TableView is a SortedList.

B) The list you provide to TableView is not a SortedList.

So in A), TableView will not do any sorting of its own, and it will not
modify its comparator (or are you talking about SortedList's
comparator?), but you can still bind the comparator to your SortedList?

And in B), will sorting happens as it currently does, i.e. by mutating
the user's list, or will TableView have an internal SortedList (as
discussed previously)?


Knut Arne Vedaa

[Jonathan Giles]

Post by Knut Arne Vedaa

Post by Jonathan Giles
The above outline essentially allows for features 2), 3), 4) and 5) from
the list above. The only feature missing therefore is 1), which is
support for returning to an unsorted state. The solution to this is to
have a special-case support inside TableView / SortedList such that the
developer may do what I mentioned above (namely,
sortedList.comparatorProperty().bind(tableView.comparatorProperty())),
and when the tableView.sortOrder list is empty the TableView /
SortedList will take that to mean there is no sort desired and to return
to the original, unsorted state.
The other important note to make is that if you as the developer provide
the TableView with a SortedList this is telling the TableView not to
sort the collection or modify the Comparator if the user / developer
changes the sortOrder (e.g. by clicking on the column headers). In other
words, a TableView with a SortedList can not have its sort order changed
by default. Of course, if you DO want the Comparator to be changed in
your SortedList (and to support unsorted state as above), you can simply
bind the comparator of the list to the tableview comparator.

This is not very clear to me. It seems that you indicate there are two
A) The list you provide to TableView is a SortedList.
B) The list you provide to TableView is not a SortedList.
So in A), TableView will not do any sorting of its own, and it will
not modify its comparator (or are you talking about SortedList's
comparator?), but you can still bind the comparator to your SortedList?

Right. To clarify: when a SortedList is set in the TableView items list
the TableView _will not_ sort it (either by actually performing a sort
or by modifying the Comparator on the SortedList). The TableView _will_
modify the TableView.comparator property to represent what the user has
requested, but this is not necessarily the guaranteed visual state.

And yes, you can bind the SortedList comparator to the TableView
comparator to get the expected sorting support.

Post by Knut Arne Vedaa
And in B), will sorting happens as it currently does, i.e. by mutating
the user's list, or will TableView have an internal SortedList (as
discussed previously)?

Mutation will happen on the users list as currently occurs - there will
be no internal SortedList.

-- Jonathan

[Knut Arne Vedaa]

Post by Jonathan Giles
Right. To clarify: when a SortedList is set in the TableView items list
the TableView _will not_ sort it (either by actually performing a sort
or by modifying the Comparator on the SortedList). The TableView _will_
modify the TableView.comparator property to represent what the user has
requested, but this is not necessarily the guaranteed visual state.
And yes, you can bind the SortedList comparator to the TableView
comparator to get the expected sorting support.

Ok, that was what I thought you meant it just wasn't quite clear from
your description. :)

Post by Jonathan Giles

Post by Knut Arne Vedaa
And in B), will sorting happens as it currently does, i.e. by mutating
the user's list, or will TableView have an internal SortedList (as
discussed previously)?

Mutation will happen on the users list as currently occurs - there will
be no internal SortedList.

How will it then be returned to an unsorted state?

And, as I'm not sure if a conclusion was reached in the previous thread
about FilteredList/SortedList, what happens here if the provided list is
a FilteredList or some other TransformedList (but not a SortedList) -
was the decision to make TransformedList mutable after all?


Knut Arne

[Jonathan Giles]

Post by Knut Arne Vedaa

Post by Jonathan Giles

Post by Knut Arne Vedaa
And in B), will sorting happens as it currently does, i.e. by mutating
the user's list, or will TableView have an internal SortedList (as
discussed previously)?

Mutation will happen on the users list as currently occurs - there will
be no internal SortedList.

How will it then be returned to an unsorted state?

It won't - when an ObservableList is used we will maintain the current
behavior of not retaining the original, unsorted state. For people who
want to support returning to an unsorted state but otherwise have
functionality like that of a normal ObservableList, they will do
something like this:

SortedList sortedList = new SortedList(SortedList.UNSORTED); // where
SortedList.UNSORTED is a special Comparator
sortedList.setAll(....);
TableView tableview = new TableView(sortedList);
sortedList.comparatorProperty().bind(tableview.comparatorProperty());

Post by Knut Arne Vedaa
And, as I'm not sure if a conclusion was reached in the previous
thread about FilteredList/SortedList, what happens here if the
provided list is a FilteredList or some other TransformedList (but not
a SortedList) - was the decision to make TransformedList mutable after
all?

I've not been closely involved in these discussions, so I'll leave that
to Richard and / or Martin to answer.

-- Jonathan

[Knut Arne Vedaa]

Post by Jonathan Giles
It won't - when an ObservableList is used we will maintain the current
behavior of not retaining the original, unsorted state. For people who
want to support returning to an unsorted state but otherwise have
functionality like that of a normal ObservableList, they will do
SortedList sortedList = new SortedList(SortedList.UNSORTED); // where
SortedList.UNSORTED is a special Comparator
sortedList.setAll(....);
TableView tableview = new TableView(sortedList);
sortedList.comparatorProperty().bind(tableview.comparatorProperty());

I see. So "return to unsorted" is thus not a feature of TableView - but
of SortedList, and the user has to set it up correctly in order to get
that feature. And providing a SortedList to TableView is a signal saying
"hey, don't sort me - I'm taking care of it on my own (and you couldn't,
even if you tried, so just don't try)".

This is not a bad solution in my opinion, although I don't think it's
100% clean to create such coupling between TableView and a specific
other class that is not appearing as a type in it's (TableView's) API.


Knut Arne

[Richard Bair]

Post by Knut Arne Vedaa
A) The list you provide to TableView is a SortedList.
B) The list you provide to TableView is not a SortedList.
So in A), TableView will not do any sorting of its own, and it will not modify its comparator (or are you talking about SortedList's comparator?), but you can still bind the comparator to your SortedList?

Right. To clarify: when a SortedList is set in the TableView items list the TableView _will not_ sort it (either by actually performing a sort or by modifying the Comparator on the SortedList). The TableView _will_ modify the TableView.comparator property to represent what the user has requested, but this is not necessarily the guaranteed visual state.
And yes, you can bind the SortedList comparator to the TableView comparator to get the expected sorting support.

And the reason for this is that the SortedList is unmodifiable (but mutable). So the add / remove methods etc throw unsupported operation exception, so from the table's perspective, it cannot be sorted. So if you use such a list, then you have to take one more step to wire up the comparator of the sorted list to the comparator on the table view, so that whenever the comparator changes the sorted list is notified of it and can do its own sorting internally.

Richard

[Knut Arne Vedaa]

Post by Richard Bair
And the reason for this is that the SortedList is unmodifiable (but mutable). So the add / remove methods etc throw unsupported operation exception, so from the table's perspective, it cannot be sorted. So if you use such a list, then you have to take one more step to wire up the comparator of the sorted list to the comparator on the table view, so that whenever the comparator changes the sorted list is notified of it and can do its own sorting internally.

What happens if you use an unmodifiable ObservableList that is not a
SortedList? Is there a way to _explicitly_ say to TableView "don't sort me"?


Knut Arne

[Jonathan Giles]

Post by Knut Arne Vedaa

Post by Richard Bair
And the reason for this is that the SortedList is unmodifiable (but
mutable). So the add / remove methods etc throw unsupported operation
exception, so from the table's perspective, it cannot be sorted. So
if you use such a list, then you have to take one more step to wire
up the comparator of the sorted list to the comparator on the table
view, so that whenever the comparator changes the sorted list is
notified of it and can do its own sorting internally.

What happens if you use an unmodifiable ObservableList that is not a
SortedList? Is there a way to _explicitly_ say to TableView "don't sort me"?

If the list is unmodifiable then it will not be sorted by TableView.
SortedList is not sorted by TableView because it is unmodifiable, not
because it is a SortedList.

-- Jonathan

[Knut Arne Vedaa]

Post by Jonathan Giles
If the list is unmodifiable then it will not be sorted by TableView.
SortedList is not sorted by TableView because it is unmodifiable, not
because it is a SortedList.

I'm confused. How does TableView know that the list is unmodifiable?


Knut Arne

[Jonathan Giles]

My understanding is that, due to historical reasons, a list can be
considered unmodifiable if, when a modification is attempted, you
receive an exception of the following types:

UnsupportedOperationException - if the add operation is not supported by
this list
ClassCastException - if the class of the specified element prevents it
from being added to this list
NullPointerException - if the specified element is null and this list
does not permit null elements
IllegalArgumentException - if some property of this element prevents it
from being added to this list

Ideally there would be a nicer way, but we have to also play nice with
what the Java collections API give us.

In other words, TableView will attempt to sort in all instances, and
will react appropriately to what it finds when this is attempted.

-- Jonathan

Post by Knut Arne Vedaa

Post by Jonathan Giles
If the list is unmodifiable then it will not be sorted by TableView.
SortedList is not sorted by TableView because it is unmodifiable, not
because it is a SortedList.

I'm confused. How does TableView know that the list is unmodifiable?
Knut Arne

[John Hendrikx]

I have a few questions, most are related to this use case:

Have a huge list (say a million items), sort it, user shift selects the
top half a million items (0-500000), now sort it again on some unrelated
criteria (making the distribution of selected items random) and now
remove those items (user presses delete). It's important none of those
actions slow the system down to a crawl (it will kill the Swing
implementation). Possible performance problems:

- Transforming a select range of 0-500000 to many smaller select ranges
when a resort occurs
- Huge 'mapping' arrays being used with "before" and "after" sort states
for keeping track of selection
- Depending on the backing list, many random deletes in a huge List will
be costly (ArrayList won't cut it, any Events being triggered on
modification of the list will likely also introduce a huge bottleneck).
Note that the use case specifically causes these deletes to be random!
It's not possible to do it in a single remove event.
- Resorting after every list change

1) Does SortedList force a specific type of List implementation, or is
it an Interface?
2) How is the change of sort order communicated to the Selection Model?
Will it be possible to override the selection model with a custom
implementation (one that is sort order independent for example) without
incurring overhead from unnecessary selection model updates caused by
sorting?

I like the flexibility provided:

SortEvent seems to cover the option of creating an (external)
implementation of sorting yourself should there be performance problems.

SortPolicy -- if I understand correctly -- externalizes the entire
sorting (and filtering?) mechanism, similar to what the SelectionModel
and FocusModel do. This keeps the View classes cleaner (they're quite
complex already) and more focused on their tasks of managing Cells and
navigation between cells. You call it a Callback, does that mean it
instantiates something or is just an interface?

The only thing for now that I think smells a bit is the special case for
SortedList. Is this hidden away in the SortPolicy or will it touch the
View classes as well?

--John

Post by Jonathan Giles
Hi all,
I am wanting to summarise my plans on how to improve TableView sorting
in JavaFX 8.0, given the conclusions reached regarding the
introduction of SortedList in JavaFX 8.0 [1]. Apologies for the length
of this email. If I had more time I would have made it shorter.
OVERVIEW
====================
For a while I've maintained an umbrella issue for improved sorting
functionality in TableView (RT-19479 [2]). My expectation is what I
outline below should work towards resolving as many of the issues
outlined in this Jira issue, most importantly the following features
1) Support returning to an unsorted state.
2) Support forcing a 'continuous' sort (so that whenever the items
list is changed the TableView remains sorted).
3) Support custom sort policies (to allow for people to override the
current in-memory sort policy and instead, say, go direct to a
database to do the sort).
4) Be notified of when a sort event is about to occur and prevent it
from happening.
5) Make it possible to force a sort when something elsewhere in the
universe (unrelated to the tableview) has changed (e.g. when you know
the tableview is no longer consistent and should re-run its sort policy).
I'm not aware of other features that should be supported. If you have
some and they are within the scope of TableView sorting, then please
read the rest of this email and then reply to it so we can discuss.
IMPLEMENTATION
====================
I have attached an early draft patch for the following functionality
inside RT-19479. It is 'early and draft' in the sense that I'm fairly
comfortable with the API, but there is still a lot of implementation
and javadoc work that needs to be done. Hopefully it should give you
all an idea of what is planned, so that you may offer any thoughts
before I get too far into implementation and javadoc.
To summarise the patch, what I've done is add a new SortEvent class,
* I've introduced a public static Callback DEFAULT_SORT_POLICY that
encapsulates the current sorting code. Users are free to overwrite
this sort policy should they wish to instead go to the database, for
example. They do this by calling tableView.setSortPolicy(...).
* I have made the TableView and TreeTableView sort() methods public,
but they simply defer to the sort policy (assuming it is non-null,
which is the case by default as the DEFAULT_SORT_POLICY is set by
default).
* I have made the code that calls sort() more full-featured. This
happens in a few places, most notably when the 'sortOrder' list is
changed (where sortOrder is a list of TableColumns which the user /
developer has specified are to be used to sort the data in the
table). In particular, I now do the following when the sortOrder
o I update the Comparator that I am about to use. This Comparator
is now publicly readable (but not writable), so that people can
do
sortedList.comparatorProperty().bind(tableView.comparatorProperty()).
+ The reason why I don't want the comparator to be writable is
that it would then allow for the comparator to be
inconsistent with the sortOrder list, whereas I view the
comparator as a reflection of the sortOrder. If people don't
want to use this comparator they are free to set their own
sort policy and use whatever they want in there.
o I fire the SortEvent. If this event is consumed I do not run the
sort().
o I call sort()
o sort() is now very simple: it calls getSortPolicy() and runs it,
passing in the TableView / TreeTableView and expecting a boolean
response to indicate success or failure.
The above outline essentially allows for features 2), 3), 4) and 5)
from the list above. The only feature missing therefore is 1), which
is support for returning to an unsorted state. The solution to this is
to have a special-case support inside TableView / SortedList such that
the developer may do what I mentioned above (namely,
sortedList.comparatorProperty().bind(tableView.comparatorProperty())),
and when the tableView.sortOrder list is empty the TableView /
SortedList will take that to mean there is no sort desired and to
return to the original, unsorted state.
The other important note to make is that if you as the developer
provide the TableView with a SortedList this is telling the TableView
not to sort the collection or modify the Comparator if the user /
developer changes the sortOrder (e.g. by clicking on the column
headers). In other words, a TableView with a SortedList can not have
its sort order changed by default. Of course, if you DO want the
Comparator to be changed in your SortedList (and to support unsorted
state as above), you can simply bind the comparator of the list to the
tableview comparator.
I hope that is clear. It's a lot of detail to get out, but hopefully
it meets the expectations of everyone and I can go forth and implement
it (and pray for no gnarly edge cases). I think the most probable area
for issues is around maintaining consistent visuals for the sorted
table column(s) given the ability for sorting to fail and / or be
ignored by the developer implementation of the comparator / sort policy.
As always, your thoughts and comments are appreciated, but lets keep
this discussion focused solely on sorting in TableView (and to a
lesser degree TreeTableView).
http://javafx-jira.kenai.com/browse/RT-17053
[2] http://javafx-jira.kenai.com/browse/RT-19479
Thanks,
-- Jonathan

[Werner Lehmann]

Other controls sometimes use a beginUpdate / endUpdate pattern to wrap
expensive operations, and to avoid e.g. 500k resorts when deleting 500k
items... Maybe that could be a solution here as well.

Post by John Hendrikx
Note that the use case specifically causes these deletes to be random!

[Knut Arne Vedaa]

Post by John Hendrikx
- Transforming a select range of 0-500000 to many smaller select ranges
when a resort occurs

Can you actually get a range out of SelectionModel, and not just a
series of indices? It doesn't seem so from the API as far as I can tell.

Post by John Hendrikx
- Depending on the backing list, many random deletes in a huge List will
be costly (ArrayList won't cut it, any Events being triggered on
modification of the list will likely also introduce a huge bottleneck).
Note that the use case specifically causes these deletes to be random!
It's not possible to do it in a single remove event.

Can't you do it in a single operation if you use List#removeAll?

Post by John Hendrikx
- Resorting after every list change

It shouldn't be necessary for SortedList to re-sort after a delete, as
the order of the remaining items stays the same.


Knut Arne

[John Hendrikx]

Post by Knut Arne Vedaa

Post by John Hendrikx
- Transforming a select range of 0-500000 to many smaller select ranges
when a resort occurs

Can you actually get a range out of SelectionModel, and not just a
series of indices? It doesn't seem so from the API as far as I can tell.

No, but the point is that the selection must be maintained after a
resort and if this involves transforming the SelectionModel (setting
different bits in a bitmap, or changing ranges) then that's quite an
expensive operation. A selection model that is sort order independent
would not need any transformation.

Post by Knut Arne Vedaa

Post by John Hendrikx
- Depending on the backing list, many random deletes in a huge List will
be costly (ArrayList won't cut it, any Events being triggered on
modification of the list will likely also introduce a huge bottleneck).
Note that the use case specifically causes these deletes to be random!
It's not possible to do it in a single remove event.

Can't you do it in a single operation if you use List#removeAll?

No, if you select the top half of the items in a list sorted by Last
Name, then resort the list by First Name, the selection will no longer
be a single range. If you decide to delete them at that point, you'll
have to do many small deletes.

Post by Knut Arne Vedaa

Post by John Hendrikx
- Resorting after every list change

It shouldn't be necessary for SortedList to re-sort after a delete, as
the order of the remaining items stays the same.

Correct, so this must be avoided (Swing didn't). Note that the
selection model in Swing was index based, so it still needed adjustment
after every delete...

Post by Knut Arne Vedaa
Knut Arne

[Scott Palmer]

Post by Knut Arne Vedaa

Post by John Hendrikx
- Depending on the backing list, many random deletes in a huge List will
be costly (ArrayList won't cut it, any Events being triggered on
modification of the list will likely also introduce a huge bottleneck).
Note that the use case specifically causes these deletes to be random!
It's not possible to do it in a single remove event.

Can't you do it in a single operation if you use List#removeAll?

No, if you select the top half of the items in a list sorted by Last Name, then resort the list by First Name, the selection will no longer be a single range. If you decide to delete them at that point, you'll have to do many small deletes.

Not with removeAll. It doesn't use a range. You give it a list containing only the nodes you wish to remove.

Scott

[Knut Arne Vedaa]

Post by Scott Palmer

Post by Knut Arne Vedaa

Post by John Hendrikx
It's not possible to do it in a single remove event.

Can't you do it in a single operation if you use List#removeAll?

No, if you select the top half of the items in a list sorted by Last Name, then resort the list by First Name, the selection will no longer be a single range. If you decide to delete them at that point, you'll have to do many small deletes.

Not with removeAll. It doesn't use a range. You give it a list containing only the nodes you wish to remove.

I think the problem is that removeAll removes by element. In a select ->
delete scenario you can't really do that because a given element may (at
least in theory) appear more than once in the list.

So for this to be efficient we would need an
ObservableList#removeAllByIndex(List<Integer> indices) method.


Knut Arne

[Alexander Kouznetsov]

Actually you can implement your own list that does index based mapping
when getting elements and then pass it to removeAll() method.

Best regards,
Alexander

Post by Knut Arne Vedaa

Post by Scott Palmer

Post by John Hendrikx

Post by Knut Arne Vedaa

Post by John Hendrikx
It's not possible to do it in a single remove event.

Can't you do it in a single operation if you use List#removeAll?

No, if you select the top half of the items in a list sorted by Last
Name, then resort the list by First Name, the selection will no
longer be a single range. If you decide to delete them at that
point, you'll have to do many small deletes.

Not with removeAll. It doesn't use a range. You give it a list
containing only the nodes you wish to remove.

I think the problem is that removeAll removes by element. In a select
-> delete scenario you can't really do that because a given element
may (at least in theory) appear more than once in the list.
So for this to be efficient we would need an
ObservableList#removeAllByIndex(List<Integer> indices) method.

[Jonathan Giles]

Post by John Hendrikx
Have a huge list (say a million items), sort it, user shift selects
the top half a million items (0-500000), now sort it again on some
unrelated criteria (making the distribution of selected items random)
and now remove those items (user presses delete). It's important none
of those actions slow the system down to a crawl (it will kill the
- Transforming a select range of 0-500000 to many smaller select
ranges when a resort occurs

I have a Jira tracking selection model performance improvements:
http://javafx-jira.kenai.com/browse/RT-10034

I don't want to delve too deeply into this (as the high-level answer is
that of course performance is critical, we need to be conscious of it,
and we have a performance team that have special performance tests that
ensure we don't have regressions here). However, in summary, at present
ListView and TreeView use a BitSet-based selection model (where each bit
represents a row where 0 = unselected and 1 = selected). TableView and
TreeTableView, on the other hand, use a slightly more archaic
ArrayList-based selection model (where the ArrayList is populated with
TablePosition instances representing a single row or row/column
position). Ideally we would transition towards a more optimised and
performant implementation as soon as possible, but I've not done
extensive research into what the best options are.

I would happily accept alternate selection model implementations if
anyone is interested in writing them :-)

Post by John Hendrikx
- Huge 'mapping' arrays being used with "before" and "after" sort
states for keeping track of selection

I'm not entirely clear what it is you're saying here.

Post by John Hendrikx
- Depending on the backing list, many random deletes in a huge List
will be costly (ArrayList won't cut it, any Events being triggered on
modification of the list will likely also introduce a huge
bottleneck). Note that the use case specifically causes these deletes
to be random! It's not possible to do it in a single remove event.
- Resorting after every list change
1) Does SortedList force a specific type of List implementation, or is
it an Interface?

I will leave this to Martin to answer.

Post by John Hendrikx
2) How is the change of sort order communicated to the Selection
Model? Will it be possible to override the selection model with a
custom implementation (one that is sort order independent for example)
without incurring overhead from unnecessary selection model updates
caused by sorting?

Selection Model watches the items list for changes and reacts
accordingly. You are more than welcome to install a custom selection
model, but then of course the onus is on you to maintain the selection
for the control.

Post by John Hendrikx
SortEvent seems to cover the option of creating an (external)
implementation of sorting yourself should there be performance problems.
SortPolicy -- if I understand correctly -- externalizes the entire
sorting (and filtering?) mechanism, similar to what the SelectionModel
and FocusModel do. This keeps the View classes cleaner (they're quite
complex already) and more focused on their tasks of managing Cells and
navigation between cells. You call it a Callback, does that mean it
instantiates something or is just an interface?

Callback is a commonly-used interface in the JavaFX world. Put simply it
is a single-method interface that accepts one argument into the method,
and returns a value. In this case we have a sortPolicy Callback that
takes a TableView (or TreeTableView) and returns a Boolean to represent
success of failure (although this API may change when I get around to
implementation).

Post by John Hendrikx
The only thing for now that I think smells a bit is the special case
for SortedList. Is this hidden away in the SortPolicy or will it
touch the View classes as well?

This is unclear yet and a topic for further discussion (although in
general it will not be something that a developer should ever concern
themselves with or necessarily even see).

-- Jonathan

[Martin Sladecek]

Post by Jonathan Giles

Post by John Hendrikx
1) Does SortedList force a specific type of List implementation, or
is it an Interface?

I will leave this to Martin to answer.

The plan is to have SortedList as a (final) class, that will wrap some
other ObservableList implementation.

-Martin