Test suite functional?

[Michael Haggerty]

Hello,

I am trying to use svnmerge.py as a library to help with a conversion of
a Subversion project to git. (git-svn understands svn:mergeinfo, but it
does not understand svnmerge.py metadata, so it needs some help.)

I would like to submit some patches to svnmerge.py that fix a problem
that I found and also help make it usable as a library. Before I
submitted the patches, I thought I should run the test suite. But...

Even with the unpatched version of svnmerge.py (from Subversion trunk
[1]), the test suite has some failures:

$ ./svnmerge_test.py
..............................................F.EE..E........
[see below for the full output]

Quite likely I'm not running the test suite correctly. For example, I
haven't compiled Subversion in the checked-out source tree; is that
necessary? (I'm running the test using the installed Subversion 1.6.17.)

Is the test suite known to be broken?

Am I doing something stupid? (Are there instructions somewhere for the
correct way to run the test suite?)

Thanks,
Michael

[1]
http://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/svnmerge
--
Michael Haggerty
mhagger-FrUbXkNCsVf2fBVCVOL8/***@public.gmane.org
http://softwareswirl.blogspot.com/

$ ./svnmerge_test.py
..............................................F.EE..E........
======================================================================
ERROR: Test that uninit works, for both merged and blocked revisions.
----------------------------------------------------------------------
File "./svnmerge_test.py", line 751, in testUninit
self.svnmerge2(["init", self.test_repo_url + "/branches/testYYY-branch"])
File "./svnmerge_test.py", line 256, in svnmerge2
ret = svnmerge.main(args)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 2361, in main
cmd(branch_dir, branch_props)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 1813, in __call__
return self.func(*args, **kwargs)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 1354, in action_init
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 942, in check_url
return get_svninfo(url) != {}
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 960, in get_svninfo
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 325, in launchsvn
return launch(cmd, **kwargs)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 280, in launch
raise LaunchError(p.returncode, cmd, stdout + stderr)
LaunchError: (1, u'svn --non-interactive info "file:///tmp/__svnmerge_test/repo/branches/testXXX-branch"', 'file:///tmp/__svnmerge_test/repo/branches/testXXX-branch: (Not a valid URL)\n\nsvn: A problem occurred; see other errors for details\n')
======================================================================
ERROR: testUninitForce (__main__.TestCase_TestRepo)
----------------------------------------------------------------------
File "./svnmerge_test.py", line 821, in testUninitForce
self.svnmerge2(["init", self.test_repo_url + "/branches/testYYY-branch"])
File "./svnmerge_test.py", line 256, in svnmerge2
ret = svnmerge.main(args)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 2361, in main
cmd(branch_dir, branch_props)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 1813, in __call__
return self.func(*args, **kwargs)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 1354, in action_init
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 942, in check_url
return get_svninfo(url) != {}
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 960, in get_svninfo
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 325, in launchsvn
return launch(cmd, **kwargs)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 280, in launch
raise LaunchError(p.returncode, cmd, stdout + stderr)
LaunchError: (1, u'svn --non-interactive info "file:///tmp/__svnmerge_test/repo/branches/testXXX-branch"', 'file:///tmp/__svnmerge_test/repo/branches/testXXX-branch: (Not a valid URL)\n\nsvn: A problem occurred; see other errors for details\n')
======================================================================
ERROR: test_invalid_url (__main__.TestCase_TestRepo)
----------------------------------------------------------------------
File "./svnmerge_test.py", line 1347, in test_invalid_url
self.assertEqual(svnmerge.get_svninfo("file://foo/bar"), {})
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 960, in get_svninfo
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 325, in launchsvn
return launch(cmd, **kwargs)
File "/home/mhagger/self/proj/subversion/trunk/contrib/client-side/svnmerge/svnmerge.py", line 280, in launch
raise LaunchError(p.returncode, cmd, stdout + stderr)
LaunchError: (1, 'svn --non-interactive info "file://foo/bar"', 'file://foo/bar: (Not a valid URL)\n\nsvn: A problem occurred; see other errors for details\n')
======================================================================
FAIL: testTransitiveMergeWithBlock (__main__.TestCase_TestRepo)
----------------------------------------------------------------------
File "./svnmerge_test.py", line 698, in testTransitiveMergeWithBlock
match=r"Committed revision 20")
File "./svnmerge_test.py", line 496, in launch
return TestCase_SvnMerge.launch(self, cmd, *args, **kwargs)
File "./svnmerge_test.py", line 306, in launch
return self._parseoutput(ret, out, **kwargs)
File "./svnmerge_test.py", line 290, in _parseoutput
"svnmerge failed, with this output:\n%s" % out)
svn: Aborting commit: '/tmp/__svnmerge_test/trunk' remains in conflict
----------------------------------------------------------------------
Ran 61 tests in 53.759s
FAILED (failures=1, errors=3)

[Alan Barrett]

Post by Michael Haggerty
I am trying to use svnmerge.py as a library to help with a
conversion of a Subversion project to git. (git-svn understands
svn:mergeinfo, but it does not understand svnmerge.py metadata,
so it needs some help.)

Perhaps it would be useful to build a filter that could
convert from svnmerge.py's metadata to svn:mergeinfo during a
dump|filter|load cycle.

--apb (Alan Barrett)

[Michael Haggerty]

Post by Michael Haggerty
I am trying to use svnmerge.py as a library to help with a conversion
of a Subversion project to git. (git-svn understands svn:mergeinfo,
but it does not understand svnmerge.py metadata, so it needs some help.)

Perhaps it would be useful to build a filter that could convert from
svnmerge.py's metadata to svn:mergeinfo during a dump|filter|load cycle.

Yes, that is a good suggestion. Such a filter would not only help
git-svn but would also help the Subversion world by "correcting" the
history in Subversion (perhaps useful for other tools that understand
svn:mergeinfo metadata). A disadvantage of this approach is that it
would not be useful for people using git-svn guerrilla fashion (against
an officially-sanctioned Subversion repository) because they wouldn't
have the rights or resources to run a dump-filter-load on the Subversion
repository.

My original idea was to hack git-svn to understand both types of merge
metadata, which would also be quite doable except for the fact that
git-svn is 6k lines of uncommented Perl code :-P

For my conversion, I used a script and a slightly hacked-up copy of
svnmerge.py to extract the metadata from the Subversion repo, then used
the information to create a bunch of git grafts and other changes to be
applied to the "first draft" git repository via git-filter-branch. In
other words, I have exhausted most of my need for my changes to
svnmerge.py, but I wanted to push them upstream for the benefit of others.

Michael

[Blair Zajac]

I am trying to use svnmerge.py as a library to help with a conversion of a Subversion project to git. (git-svn understands svn:mergeinfo, but it does not understand svnmerge.py metadata, so it needs some help.)

Perhaps it would be useful to build a filter that could convert from svnmerge.py's metadata to svn:mergeinfo during a dump|filter|load cycle.

There are two tools that convert from svnmerge.py to svn:mergeinfo, look in here for svnmerge-migrate*

https://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/svnmerge

Blair

[Michael Haggerty]

Post by Blair Zajac

I am trying to use svnmerge.py as a library to help with a conversion of a Subversion project to git. (git-svn understands svn:mergeinfo, but it does not understand svnmerge.py metadata, so it needs some help.)

Perhaps it would be useful to build a filter that could convert from svnmerge.py's metadata to svn:mergeinfo during a dump|filter|load cycle.

There are two tools that convert from svnmerge.py to svn:mergeinfo, look in here for svnmerge-migrate*
https://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/svnmerge

Probably you mean that I look at these tools for inspiration, and that
is a good suggestion. But to avoid that somebody thinks that these
tools solve the problem, let me explain why it is not the case.

The svnmerge-migrate* scripts only migrate the metadata at HEAD,
creating a new commit for each branch that contains the svn:mergeinfo
equivalent of the svnmerge metadata that was present in HEAD. The
result is that--to a tool that understands svn:mergeinfo but not
svnmerge.py metadata--it looks like that new commit includes all of the
merges that were ever done using svnmerge.py.

A history-aware data migration requires historical merges to be
represented at the time they were made in the history. That is why we
definitely need a tool that rewrites the past; either in the form of a
dump-filter-rewrite of the Subversion repository, or in the form of a
tool that understands the history while migrating to another VCS.

Michael

[Blair Zajac]

Post by Michael Haggerty

Post by Blair Zajac

Post by Michael Haggerty
I am trying to use svnmerge.py as a library to help with a
conversion of a Subversion project to git. (git-svn understands
svn:mergeinfo, but it does not understand svnmerge.py metadata,
so it needs some help.)

Perhaps it would be useful to build a filter that could convert
from svnmerge.py's metadata to svn:mergeinfo during a dump|filter|
load cycle.

There are two tools that convert from svnmerge.py to svn:mergeinfo,
look in here for svnmerge-migrate*
https://svn.apache.org/repos/asf/subversion/trunk/contrib/client-side/svnmerge

Probably you mean that I look at these tools for inspiration, and that
is a good suggestion. But to avoid that somebody thinks that these
tools solve the problem, let me explain why it is not the case.
The svnmerge-migrate* scripts only migrate the metadata at HEAD,
creating a new commit for each branch that contains the svn:mergeinfo
equivalent of the svnmerge metadata that was present in HEAD. The
result is that--to a tool that understands svn:mergeinfo but not
svnmerge.py metadata--it looks like that new commit includes all of the
merges that were ever done using svnmerge.py.
A history-aware data migration requires historical merges to be
represented at the time they were made in the history. That is why we
definitely need a tool that rewrites the past; either in the form of a
dump-filter-rewrite of the Subversion repository, or in the form of a
tool that understands the history while migrating to another VCS.

Good points, thanks for the explanation.

For complicated svn installs, the later is definitely safer. Any bugs
in understanding the history could be managed by re-cloning part of
svn's repository, instead of a dump-filter-rewrite that would require
everyone to do fresh checkouts.

Blair