Version control for collaboration (with word-level diffs)?

Question

Most papers are now written collaboratively, and collaborators are often located in different places. I have always used version control systems for my documents and code, and have also found version control critical for collaborative software projects, but it seems many researchers in theory avoid their use for writing joint papers. To convince my collaborators that version control (revision control) is a good idea for working together, there seem to be some prerequisites. It is not possible to force everyone to worry about a specific set of conventions for line breaks and paragraphs, or to avoid tab/space conversions.

Does someone offer free hosting of small shared document repositories, with text-document-friendly version control that can handle word-level diffs (not line-based)?

If not, then I would welcome other suggestions that are based on experience (let's avoid speculation, please).

I was thinking of Git, Subversion, Mercurial, darcs, or Bazaar, set up to handle word-level differences with wdiff, together with a simple way of setting up access secured by public keys (for instance via ssh). However, none of the version control providers that I looked at seem to offer anything like this. For scientific collaboration the "enterprise" features stressed by many of these companies are not very important (lots of branches, integration with trac, auditing by third parties, hierarchical project teams). But word-level diffs seem critical yet unsupported. In my experience, with line-level diffs for text files, everyone has to avoid reformatting paragraphs and editors that change tabs to spaces or vice versa cause problems; there also seem to be many spurious edit conflicts. I have used wdiff and latexdiff quite successfully to help with manual merging of changes previously, so I am hoping a word-level diff would reduce such problems.

See related question at MO about tools for collaboration, and related questions over at TeX.SE, about version control for LaTeX documents and LaTeX packages for version control. See also the SVN Hosting Comparison Review Chart for a large list of hosting providers, for just one of the main version control systems.

---

Edit: Jukka Suomela's answer to the TeX.SE question "Best LaTeX-aware diff and merge tools for subversion" seems to be the best suggestion so far, covering how to interpret the deltas on a word level. Moreover, Jukka has explained how the differences between successive versions on the repository end are separate from the user-level differences used for conflict detection and merging of changes. Jukka's answer at TeX.SE explicitly excludes simultaneous edits and merging, relying instead on the traditional atomic edit token to avoid edit conflicts. Clarifying (and modifying) my original question, is there a way to ensure that edit conflicts can be resolved on a word difference basis, rather than on a line difference basis? In other words, can wdiff or similar tools be integrated into the conflict detection part of the version control tools, similar to the way end-of-line differences and differences in whitespace can be ignored?

Answer 1

I've used git to collaborate on some documents written in latex. You'll have to adhere to some rules:

• Start each sentence on a new line, latex ignores these newlines as long as there is no blank line
• Use the same configuration for formatting (tab/spaces/max text width)
• For best results, create a .gitattributes file in your repository and add the line *.tex diff=tex. This makes diff aware of the tex syntax and leads to more meaningful output.

You can then use git diff --color-words and gitk --color-words to see word differences (also see this article Word-by-word diffs in Git on how to configure git to always use word-diff algorithm for displaying the git diff/git log).

To reduce the manual merges, I can recommend using separate files for sections and subsections (depending on the size of your document).

Answer 2

I suggest taking a look at:

LaTeX/Collaborative Writing of LaTeX Documents,

and the related paper:

Tools for Collaborative Writing of Scientific LaTeX Documents.

Answer 3

I really want to echo others and suggest that you sit down and work out a nice SVN strategy. I use SVN to host my entire "research" structure:

• JabRef reference managing
• Downloaded PDFs
• Articles

It's great because it contains everything, and of course provides a history. The caveat being you need your own server. But if you have some existing Windows machine (or whatever you a comfortable with) you can install it simply through VisualSVN Server. You then create appropriate accounts for collaborators, and give them access to an appropriate area (i.e. perhaps read-access to your JabRef bibtex file, and read/write to a shared 'in-progress' article area).

TortiseSVN can be used as the Windows client for interacting with SVN. You need to be careful about moving/deleting files and copying folders (SVN will store metadata inside hidden folders in each of your folders, so you must execute the delete command from within SVN to get rid of it, it takes a bit of getting used to, but is worth the investment).

Then, when working with a collaborator, they clearly must also use SVN. But, again, the investment in learning is not worthless. And via some thought, you can also have it so you have read-only access to their jabref file (perhaps via the 'external' facility in svn).

In this way, with a bit of thought and a little bit of effort, you can be in a situation where you are editing documents as per normal, commiting changes nightly, updating in the morning and resolving all conflicts easily.

I really recommend it. The more people that set up their own SVNs the better, as it will only improve collaboration options in the future (though, of course, it would be beneficial if perhaps there was a 'standard' way of setting up a scientific repository).

-- Edit: Infact, I've written up such a proposal here: Strategy for Scientific Collaboration with LaTeX and SVN. It proposes to make use of the svn externals feature to allow easy collaboration between people with a similar setup. Let me know if it needs changing or is simply not appropriate.

Answer 4

While reading your great post and looking around for a solution myself I stumbled into the option to colorize changes at word level in gitk. The gitk parameter seems to be a new and/or undocumented feature since the auto-completion does not offer it and the gitk man page does not list it.
Here are the options the I found:

gitk --word-diff=plain
gitk --word-diff=porcelain
gitk --word-diff=color

You can find several discussions on that topic searching for "diff --color-words" gitk.

Edit:
This is what is looks like ...

Answer 5

I understand the problem very well. I have started using Kaleidoscope for diffs with git. It is Mac-only but its comparisons work better than wdiff, and it also has an interface and live updates.