Started refactoring of the documentation towards smaller chapters, grouped into logical units with better explanations (hopefully).

en/index.rst

@@ -63,10 +63,15 @@ Working with Objects
   :doc:`QueryBuilder <reference/query-builder>` |
   :doc:`Native SQL Queries <reference/native-sql>`
 
+* **UnitOfWork dissected**:
+  :doc:`Doctrine Internals explained <reference/unitofwork>` |
+  :doc:`Owning and Inverse Side Associations <reference/unitofwork-associations>` 
+
 * **Tutorials**:
   :doc:`Indexed associations <tutorials/working-with-indexed-associations>` |
   :doc:`Extra Lazy Assocations <tutorials/extra-lazy-associations>` |
-  :doc:`Composite Primary Keys <tutorials/composite-primary-keys>` 
+  :doc:`Composite Primary Keys <tutorials/composite-primary-keys>` |
+  :doc:`Ordered associations <tutorials/ordered-associations>`
 
 Advanced Topics
 ---------------

en/reference/limitations-and-known-issues.rst

@@ -76,47 +76,6 @@ objects using ``serialize()/deserialize()`` which the DBAL Type
 The feature request for full value-object support
 `is described in the DDC-93 ticket <http://www.doctrine-project.org/jira/browse/DDC-93>`_.
 
-Applying Filter Rules to any Query
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-There are scenarios in many applications where you want to apply
-additional filter rules to each query implicitly. Examples
-include:
-
-
--  In I18N Applications restrict results to a entities annotated
-   with a specific locale
--  For a large collection always only return objects in a specific
-   date range/where condition applied.
--  Soft-Delete
-
-There is currently no way to achieve this consistently across both
-DQL and Repository/Persister generated queries, but as this is a
-pretty important feature we plan to add support for it in the
-future.
-
-Restricing Associations
-~~~~~~~~~~~~~~~~~~~~~~~
-
-There is currently no way to restrict associations to a subset of entities matching a given condition.
-You should use a DQL query to retrieve a subset of associated entities. For example
-if you need all visible articles in a certain category you could have the following code
-in an entity repository:
-
-.. code-block:: php
-
-    <?php
-    class ArticleRepository extends EntityRepository
-    {
-        public function getVisibleByCategory(Category $category)
-        {
-            $dql = "SELECT a FROM Article a WHERE a.category = ?1 and a.visible = true";
-            return $this->getEntityManager()
-                        ->createQuery($dql)
-                        ->setParameter(1, $category)
-                        ->getResult();
-        }
-    }
 
 Cascade Merge with Bi-directional Associations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -162,7 +121,7 @@ to the same entity.
 Behaviors
 ~~~~~~~~~
 
-Doctrine 2 *will never* include a behavior system like Doctrine 1
+Doctrine 2 will **never** include a behavior system like Doctrine 1
 in the core library. We don't think behaviors add more value than
 they cost pain and debugging hell. Please see the many different
 blog posts we have written on this topics:
@@ -173,7 +132,7 @@ blog posts we have written on this topics:
 -  `Doctrine 2 Behavioral Extensions <http://www.doctrine-project.org/blog/doctrine2-behavioral-extensions>`_
 -  `Doctrator <https://github.com/pablodip/doctrator`>_
 
-Doctrine 2 has enough hooks and extension points so that *you* can
+Doctrine 2 has enough hooks and extension points so that **you** can
 add whatever you want on top of it. None of this will ever become
 core functionality of Doctrine2 however, you will have to rely on
 third party extensions for magical behaviors.
@@ -199,11 +158,15 @@ backwards compatibility issues or where no simple fix exists (yet).
 We don't plan to add every bug in the tracker there, just those
 issues that can potentially cause nightmares or pain of any sort.
 
+See the Open Bugs on Jira for more details on `bugs, improvement and feature
+requests
+<http://www.doctrine-project.org/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&pid=10032&resolution=-1&sorter/field=updated&sorter/order=DESC>`_.
+
 Identifier Quoting and Legacy Databases
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 For compatibility reasons between all the supported vendors and
-edge case problems Doctrine 2 does *NOT* do automatic identifier
+edge case problems Doctrine 2 does **NOT** do automatic identifier
 quoting. This can lead to problems when trying to get
 legacy-databases to work with Doctrine 2.
 
@@ -222,5 +185,5 @@ tables and column names to avoid the legacy quoting nightmare.
 Microsoft SQL Server and Doctrine "datetime"
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Doctrine assumes that you use DateTime2 data-types. If your legacy database contains DateTime
+Doctrine assumes that you use ``DateTime2`` data-types. If your legacy database contains DateTime
-datatypes then you have to add your own data-type (see Basic Mapping for an example).
+datatypes then you have to add your own data-type (see Basic Mapping for an example).

en/reference/unitofwork-associations.rst

@@ -0,0 +1,60 @@
+Association Updates: Owning Side and Inverse Side
+=================================================
+
+When mapping bidirectional associations it is important to
+understand the concept of the owning and inverse sides. The
+following general rules apply:
+
+-  Relationships may be bidirectional or unidirectional.
+-  A bidirectional relationship has both an owning side and an inverse side
+-  A unidirectional relationship only has an owning side.
+-  Doctrine will **only** check the owning side of an assocation for changes.
+
+Bidirectional Associations
+--------------------------
+
+The following rules apply to **bidirectional** associations:
+
+- The inverse side has to use the ``mappedBy`` attribute of the OneToOne,
+  OneToMany, or ManyToMany mapping declaration. The mappedBy
+  attribute contains the name of the association-field on the owning side.
+- The owning side has to use the ``inversedBy`` attribute of the
+  OneToOne, ManyToOne, or ManyToMany mapping declaration. 
+  The inversedBy attribute contains the name of the association-field
+  on the inverse-side.
+- ManyToOne is always the owning side of a bidirectional assocation.
+- OneToMany is always the inverse side of a bidirectional assocation.
+- The owning side of a OneToOne assocation is the entity with the table
+  containing the foreign key.
+- You can pick the owning side of a many-to-many assocation yourself.
+
+Important concepts
+------------------
+
+**Doctrine will only check the owning side of an assocation for changes.**
+
+To fully understand this, remember how bidirectional associations
+are maintained in the object world. There are 2 references on each
+side of the association and these 2 references both represent the
+same association but can change independently of one another. Of
+course, in a correct application the semantics of the bidirectional
+association are properly maintained by the application developer
+(that's his responsibility). Doctrine needs to know which of these
+two in-memory references is the one that should be persisted and
+which not. This is what the owning/inverse concept is mainly used
+for.
+
+**Changes made only to the inverse side of an association are ignored. Make sure to update both sides of a bidirectional association (or at least the owning side, from Doctrine's point of view)**
+
+The owning side of a bidirectional association is the side Doctrine
+"looks at" when determining the state of the association, and
+consequently whether there is anything to do to update the
+association in the database.
+
+.. note::
+
+    "Owning side" and "inverse side" are technical concepts of
+    the ORM technology, not concepts of your domain model. What you
+    consider as the owning side in your domain model can be different
+    from what the owning side is for Doctrine. These are unrelated.
+

en/reference/unitofwork.rst

@@ -0,0 +1,135 @@
+Doctrine Internals explained
+============================
+
+Object relational mapping is a complex topic and sufficiently understanding how Doctrine works internally helps you use its full power.
+
+How Doctrine keeps track of Objects
+-----------------------------------
+
+Doctrine uses the Identity Map pattern to track objects. Whenever you fetch an
+object from the database, Doctrine will keep a reference to this object inside
+its UnitOfWork. The array holding all the entity references is two-levels deep
+and has the keys "root entity name" and "id". Since Doctrine allows composite
+keys the id is a sorted, serialized version of all the key columns.
+
+This allows Doctrine room for optimizations. If you call the EntiyManager and
+ask for an entity with a specific ID twice, it will return the same instance:
+
+.. code-block:: php
+
+    public function testIdentityMap()
+    {
+        $objectA = $this->entityManager->find('EntityName', 1);
+        $objectB = $this->entityManager->find('EntityName', 1);
+
+        $this->assertSame($objectA, $objectB)
+    }
+
+Only one SELECT query will be fired against the database here. In the second
+``EntityManager#find()`` call Doctrine will check the identity map first and
+doesn't need to make that database roundtrip.
+
+Even if you get a proxy object first then fetch the object by the same id you
+will still end up with the same reference:
+
+.. code-block:: php
+
+    public function testIdentityMapReference()
+    {
+        $objectA = $this->entityManager->getReference('EntityName', 1);
+        // check for proxyinterface
+        $this->assertInstanceOf('Doctrine\ORM\Proxy\Proxy', $objectA);
+
+        $objectB = $this->entityManager->find('EntityName', 1);
+
+        $this->assertSame($objectA, $objectB)
+    }
+
+The identity map being indexed by primary keys only allows shortcuts when you
+ask for objects by primary key. Assume you have the following ``persons``
+table:
+
+::
+
+    id | name
+    -------------
+    1  | Benjamin
+    2  | Bud
+
+Take the following example where two
+consecutive calls are made against a repository to fetch an entity by a set of
+criteria:
+
+.. code-block:: php
+
+    public function testIdentityMapRepositoryFindBy()
+    {
+        $repository = $this->entityManager->getRepository('Person');
+        $objectA = $repository->findOneBy(array('name' => 'Benjamin'));
+        $objectB = $repository->findOneBy(array('name' => 'Benjamin'));
+
+        $this->assertSame($objectA, $objectB);
+    }
+
+This query will still return the same references and `$objectA` and `$objectB`
+are indeed referencing the same object. However when checking your SQL logs you
+will realize that two queries have been executed against the database. Doctrine
+only knows objects by id, so a query for different criteria has to go to the
+database, even if it was executed just before.
+
+But instead of creating a second Person object Doctrine first gets the primary
+key from the row and check if it already has an object inside the UnitOfWork
+with that primary key. In our example it finds an object and decides to return
+this instead of creating a new one.
+
+The identity map has a second use-case. When you call ``EntityManager#flush``
+Doctrine will ask the identity map for all objects that are currently managed.
+This means you don't have to call ``EntityManager#persist`` over and over again
+to pass known objects to the EntityManager. This is a NO-OP for known entities,
+but leads to much code written that is confusing to other developers.
+
+The following code WILL update your database with the changes made to the
+``Person`` object, even if you did not call ``EntityManager#persist``:
+
+.. code-block:: php
+
+    <?php
+    $user = $entityManager->find("Person", 1);
+    $user->setName("Guilherme");
+    $entityManager->flush();
+
+How Doctrine Detects Changes
+----------------------------
+
+Doctrine is a data-mapper that tries to achieve persistence-ignorance (PI).
+This means you map php objects into a relational database that don't
+necessarily know about the database at all. A natural question would now be,
+"how does Doctrine even detect objects have changed?". 
+
+For this Doctrine keeps a second map inside the UnitOfWork. Whenever you fetch
+an object from the database Doctrine will keep a copy of all the properties and
+associations inside the UnitOfWork. Because variables in the PHP language are
+subject to "copy-on-write" the memory usage of a PHP request that only reads
+objects from the database is the same as if Doctrine did not keep this variable
+copy. Only if you start changing variables PHP will create new variables internally
+that consume new memory.
+
+Now whenever you call ``EntityManager#flush`` Doctrine will iterate over the
+Identity Map and for each object compares the original property and association
+values with the values that are currently set on the object. If changes are
+detected then the object is qeued for an SQL UPDATE operation. Only the fields
+that actually changed are updated.
+
+This process has an obvious performance impact. The larger the size of the
+UnitOfWork is, the longer this computation takes. There are several ways to
+optimize the performance of the Flush Operation:
+
+- Mark entities as read only. These entities can only be inserted or removed,
+  but are never updated. They are omitted in the changeset calculation.
+- Temporarily mark entities as read only. If you have a very large UnitOfWork
+  but know that a large set of entities has not changed, just mark them as read
+  only with ``$entityManager->getUnitOfWork()->markReadOnly($entity)``.
+- Flush only a single entity with ``$entityManager->flush($entity)``.
+- Use :doc:`Change Tracking Policies <change-tracking-policies>` to use more
+  explicit strategies of notifying the UnitOfWork what objects/properties
+  changed.

en/reference/working-with-associations.rst

@@ -7,7 +7,7 @@ collections of objects. When it comes to persistence, it is
 important to understand three main things:
 
 
--  The :ref:`concept of owning and inverse sides <association-mapping-owning-inverse>`
+-  The :doc:`concept of owning and inverse sides <unitofwork-associations>`
    in bidirectional associations.
 -  If an entity is removed from a collection, the association is
    removed, not the entity itself. A collection of entities always

en/toc.rst

@@ -11,6 +11,7 @@ Tutorials
    tutorials/working-with-indexed-associations
    tutorials/extra-lazy-associations
    tutorials/composite-primary-keys
+   tutorials/ordered-associations
 
 Reference Guide
 ---------------
@@ -28,8 +29,10 @@ Reference Guide
    reference/inheritance-mapping
    reference/working-with-objects
    reference/working-with-associations
-   reference/transactions-and-concurrency
    reference/events
+   reference/unitofwork
+   reference/unitofwork-associations
+   reference/transactions-and-concurrency
    reference/batch-processing
    reference/dql-doctrine-query-language
    reference/query-builder
@@ -48,7 +51,6 @@ Reference Guide
    reference/limitations-and-known-issues
 
 
-
 Cookbook
 --------
 

en/tutorials/ordered-associations.rst

@@ -0,0 +1,79 @@
+Ordering To-Many Assocations
+----------------------------
+
+There use-cases you will want to sort collections when they are
+retrieved from the database. In userland you do this as long as you
+haven't initially saved an entity with its associations into the
+database. To retrieve a sorted collection from the database you can
+use the ``@OrderBy`` annotation with an collection that specifies
+an DQL snippet that is appended to all queries with this
+collection.
+
+Additional to any ``@OneToMany`` or ``@ManyToMany`` annotation you
+can specify the ``@OrderBy`` in the following way:
+
+.. code-block:: php
+
+    <?php
+    /** @Entity **/
+    class User
+    {
+        // ...
+    
+        /**
+         * @ManyToMany(targetEntity="Group")
+         * @OrderBy({"name" = "ASC"})
+         **/
+        private $groups;
+    }
+
+The DQL Snippet in OrderBy is only allowed to consist of
+unqualified, unquoted field names and of an optional ASC/DESC
+positional statement. Multiple Fields are separated by a comma (,).
+The referenced field names have to exist on the ``targetEntity``
+class of the ``@ManyToMany`` or ``@OneToMany`` annotation.
+
+The semantics of this feature can be described as follows.
+
+
+-  ``@OrderBy`` acts as an implicit ORDER BY clause for the given
+   fields, that is appended to all the explicitly given ORDER BY
+   items.
+-  All collections of the ordered type are always retrieved in an
+   ordered fashion.
+-  To keep the database impact low, these implicit ORDER BY items
+   are only added to an DQL Query if the collection is fetch joined in
+   the DQL query.
+
+Given our previously defined example, the following would not add
+ORDER BY, since g is not fetch joined:
+
+.. code-block:: sql
+
+    SELECT u FROM User u JOIN u.groups g WHERE SIZE(g) > 10
+
+However the following:
+
+.. code-block:: sql
+
+    SELECT u, g FROM User u JOIN u.groups g WHERE u.id = 10
+
+...would internally be rewritten to:
+
+.. code-block:: sql
+
+    SELECT u, g FROM User u JOIN u.groups g WHERE u.id = 10 ORDER BY g.name ASC
+
+You can't reverse the order with an explicit DQL ORDER BY:
+
+.. code-block:: sql
+
+    SELECT u, g FROM User u JOIN u.groups g WHERE u.id = 10 ORDER BY g.name DESC
+
+...is internally rewritten to:
+
+.. code-block:: sql
+
+    SELECT u, g FROM User u JOIN u.groups g WHERE u.id = 10 ORDER BY g.name DESC, g.name ASC
+
+