MOSS MVP

I've moved my blog to http://blog.falchionconsulting.com!. Please update your links. This blog is no longer in use--you can find all posts and comments at my new blog; I will no longer be posting to this site and comments have been disabled.

Wednesday, September 12, 2007

Import/Export/Copy Lists

One of the challenges I'm currently trying to solve is related to the site directory. After my upgrade the site directory is a bit messed up (the upgrade seemed to have merged columns). In an attempt to get the site directory in a state that I can work with I'm looking at several possibilities. One of those was configure the site directory the way I want it, export the Sites list and then import back after my upgrade completes. In order to accomplish this I had to create the ability to import and export lists (turns out this alone won't solve my problems but the commands I've created could be extremely handy in many scenarios).

Stsadm has the ability out of the box to export and import web sites but it doesn't support doing just lists. However, the content deployment API does allow you to import and export any object type (right down to an individual list item). So the solution was rather simple - I just created two new commands which function almost identically to the existing import and export commands except that they work with lists (export takes in an URL representing a view of the list and import takes in a target web to import the list in to).

I also created a copy command which wraps up the import and export so that both can be done in one step (passing "-deletesource" to the copylist command will make it function like a move command). While working on all this I also decided that I got tired of writing the same parameter validation code over and over again so I created a new base class (SPOperation) and some parameter classes (SPParam and SPParamCollection). These classes use several validator classes that help with more complex validations. Note however that I didn't re-invent the wheel here - these classes are all almost straight copies of what Microsoft uses internally in the stsadm.exe assembly itself (if you use Reflector to look at stsadm.exe you'll see these classes defined there - I didn't want to deal with lots of reflection to use these so I just recreated them thereby allowing me to make modifications to suite my own needs where necessary).

The code to work with the content deployment API is pretty straightforward. Stefan Goßner does a really good job at explaining how to use the API in his blog Deep Dive into SharePoint Content Deployment and Migration API so I won't reiterate it here. The commands I created are detailed below.

1. gl-exportlist

This command has almost exactly the same parameters as the built in "export" command. The only thing of note is that the default value for the versions flag is 4 (or All versions for files and list items) in mine whereas the default for the export operation is 1 (or Last major version for files and list items). I went back and forth on whether I should keep it the same but decided in the end that for my purposes when I want to copy a list I usually want to retain everything so I chose that as the default. I also considered whether I wanted the caller to pass in the list name or GUID or something else and in the end I decided that the easiest thing would be to use the URL to a view.

I created a simple helper method which takes this passed in url and searches all lists and all views until it finds a match (there's probably an easier way to do this but this works and I didn't have a lot of time to mess with it). The syntax of the command can be seen below:

c:\>stsadm -help gl-exportlist

stsadm -o gl-exportlist


Exports a list.

Parameters:
        -url <list view url>
        -filename <export file name>
        [-overwrite]
        [-includeusersecurity]
        [-haltonwarning]
        [-haltonfatalerror]
        [-nologfile]
        [-versions <1-4>
                1 - Last major version for files and list items
                2 - The current version, either the last major or the last minor
                3 - Last major and last minor version for files and list items
                4 - All versions for files and list items (default)]
        [-cabsize <integer from 1-1024 megabytes> (default: 25)]
        [-nofilecompression]
        [-includedescendants <All | Content | None>]
        [-excludedependencies (Specifies whether to exclude dependencies from the export package when exporting objects of type SPFile or SPListItem)]
        [-quiet]

Here’s an example of how to export the Documents list from the root site collection:

stsadm –o gl-exportlist -url "http://intranet/Documents/Forms/AllItems.aspx" -filename "c:\docs" -includeusersecurity -versions 4 -nofilecompression

Running the above will create a folder called "docs" in the root c: drive. You'll also find a log file there containing the same information that was dumped to the console. This same output can then be used by the gl-importlist command to copy the list to another site collection (note that the target could be on any web app in any farm).

Update 11/3/2008: I've added a new option, excludedependencies, which allows you to exclude exporting dependent items - this will get around a lot of issues that some people have had when trying to import and were receiving errors about dependent items not existing.

2. gl-importlist

This command has almost exactly the same parameters as the built in "import" command. One additional parameter of note is the "-retargetlinks" parameter. If this parameter is specified then you must also specify the "-sourceurl" parameter which corresponds to a view of the source list. The "retargetlinks" parameter tells the code to find any links that point to items in the source list and make them point to the target list. This becomes helpful in a move operation. Note that only items identified by the list items BackwardLinks collection will get modified (unfortunately I'm not 100% clear about when and how this collection is set so I don't claim that using this will fix every link to your items, though I am planning on creating another command that will loop through every field of every item of every list of every site and adjust the field value by replacing one value with another). The syntax of the command can be seen below:

C:\>stsadm -help gl-importlist

stsadm -o gl-importlist

Imports a list.

Parameters:
        -url <url of a web site to import to>
        -filename <import file name>
        [-includeusersecurity]
        [-haltonwarning]
        [-haltonfatalerror]
        [-nologfile]
        [-updateversions <1-3>
                1 - Add new versions to the current file (default)
                2 - Overwrite the file and all its versions (delete then insert)
                3 - Ignore the file if it exists on the destination]
        [-nofilecompression]
        [-quiet]
        [-retargetlinks (resets links pointing to the source to now point to the target)]
        [-sourceurl <url to a view of the original list> (use if retargetlinks)]
        [-retainobjectidentity]
        [-copysecuritysettings (must provide sourceurl and includeusersecurity)]

Here’s an example of how to import the Documents list that we exported above and re-target any links pointing to the original documents so that they now point to the new documents (note that just like the built in import operation if you specify nofilecompression for the export you must also specify it for the import):

stsadm –o gl-importlist -url "http://teamsites/" -filename "c:\docs" -includeusersecurity -updateversions 2 -nofilecompression -retargetlinks -sourceurl "http://intranet/Documents/Forms/AllItems.aspx"

As with the gl-exportlist operation a log file will be generated which will include all the details that were dumped to the console.

Update 10/8/2007: I've added a retainobjectidentity identity parameter to the gl-importlist command which allows you keep all IDs the same after the import. Note that the source cannot exist in the content database or you will get an error using this switch.

3. gl-copylist

If you've got the ability to import and export a list then why not make it a one step operation to copy a list or move a list. Once I had the code for the import and export the copy was nothing more than a wrapper which would call into the appropriate methods. By adding an extra parameter to support deleting the source list I now also had a move command. The syntax of the command is just the combination of the import and export minus that which I don't need the user to set. The syntax of the command can be seen below:

C:\>stsadm -help gl-copylist

Copies a list to a new web.

Parameters:
        -sourceurl <list view url to copy>
        -targeturl <url of a web site to copy to>
        [-includeusersecurity]
        [-haltonwarning]
        [-haltonfatalerror]
        [-nologfile]
        [-versions <1-4>
                1 - Last major version for files and list items
                2 - The current version, either the last major or the last minor
                3 - Last major and last minor version for files and list items
                4 - All versions for files and list items (default)]
        [-updateversions <1-3>
                1 - Add new versions to the current file (default)
                2 - Overwrite the file and all its versions (delete then insert)

                3 - Ignore the file if it exists on the destination]
        [-quiet]
        [-retargetlinks (resets links pointing to the source to now point to the target)]
        [-deletesource]
        [-temppath <temporary folder path for storing of export files>]
        [-includedescendants <All | Content | None>]
        [-excludedependencies (Specifies whether to exclude dependencies from the export package when exporting objects of type SPFile or SPListItem)]

Here’s an example of how to perform the same operation that the above import and export commands were doing but with just one step while at the same time deleting the source list:

stsadm –o gl-copylist -sourceurl "http://intranet/Documents/Forms/AllItems.aspx" -targeturl "http://teamsites/" -includeusersecurity -updateversions 2 -versions 4 -retargetlinks

If the source list was deleted then the result of the export operation are saved to a temp folder, the path to which is displayed to the user upon completion.

Update 11/3/2008: I've added a new option, excludedependencies, which allows you to exclude exporting dependent items - this will get around a lot of issues that some people have had when trying to import and were receiving errors about dependent items not existing.  I've also added the includedescendants parameter to the copy command to bring it in line with the export command.

4. gl-getlistschemaxml

This is a command that I really only created to help me debug something. I was going to delete it but figured I'd keep it around in case someone found a use for it. All the command does is grab an instance of the SPList object corresponding to the view URL that was passed in and then outputs the SchemaXml property (using an XmlTextWriter to provide formatting). Maybe someone will find a use for it (perhaps in creating a feature definition file or something). The syntax of the command can be seen below:

C:\>stsadm -help gl-getlistschemaxml

Returns the XML schema for a list.

Parameters:
        -url <list view url>

Here’s an example of how to get the schema XML for the Documents list:

stsadm –o gl-getlistschemaxml -url "http://intranet/Documents/Forms/AllItems.aspx"

The output for this is rather lengthy so I won't bother showing it here.

Update 11/16/2007: I've updated the gl-copylist and gl-importlist commands to now support the functionality of the gl-copylistsecurity command. The content above has been updated to reflect this change where appropriate. Note that for this to work for the gl-importlist command the sourceurl and includeusersecurity parameters must be specified.

169 comments:

Mike Smith said...

Really great extensions here! Thanks!

I found one problem with the exportlist...

You say the option -file "C:\docs" will create a folder called "docs" in the root c: drive.

Instead it creates two files starting with "docs": docs.cmp and docs.export.log.

Easy enough to fix, just use: -file "C:\docs\somefilename". This will create the docs directory and then the two files inside of it.

Gary Lapointe said...

Mike - thanks for the feedback - actually the export is working correctly (works just like the stsadm export command) - I might have just not explained it fully. If you pass in nofilecompression then it will create a folder but if you don't pass that flag in then it just appends the cmp extension to whatever is passed in. I tried to keep the functionality exactly the same as the built in export function so that it would be familiar to those that have used it. Hope that clarifies it and again, thanks for the feedback.

Gianbattista said...

the export/import work perfectly with standard list but with a custom list it's like a copy (the modified date and the user are the user that execute the stsadm commands)

any idea?

Gary Lapointe said...

I'm not sure I'm following what you are asking. Can you elaborate?

Gianbattista said...

if I've a custom list with a document modified by user "A" and the user "B" launchs the stsadm exportlist and importlist, the document in the imported list results created and modified by the user "B".

the commands are

stsadm -o exportlist -url "custom list url" -filename "filename" -versions 4

and

stsadm -o importlist -url "destination url " -filename "filename" -updateversions 1


thanks in advance for any help

Gary Lapointe said...

I see what you're saying now. Unfortunately this isn't something I'm able to fix (at least not easily) as it's a function of the deployment API from Microsoft. All I'm doing is using their API to do the import. I'd have to either rewrite their import API or write some other routine to attempt to change the modified by user after the import occurs (which can be more problematic that it might seem). You'll find the same thing will happen if you use the out of the box import command to import an entire web. I'm sorry I'm not being more helpful on this one - I'd suggest giving your feedback to Microsoft and hopefully if enough people complain they'll make it an option in the future (this hasn't been an issue for my company so I won't have time to do anything on my end unless it becomes an issue for my company).

Gianbattista said...

Thank
adding the -includeusersecurity it seems that the bug is fixed.
Anyway there is anothe problem: the versions history for documents is wrong. Example
file1 whith
v2 By Author "A" date "dateX"
v1 By Author "B" data "DateY"
after export/import in the new list become:
v2 By Author "A" date "DateX"
v1 By Author "A" date "DateX"

this happens also with standard stsadm import/export.... I think also this behavior is "by design" for the MS API :-)

Thanks in advance

Gary Lapointe said...

I guess that makes sense that the includeusersecurity flag would affect that (didn't think to consider it). Not sure what's up with the version info but you're right that it's a Microsoft thing.

Adam said...

Hi Gary this looks great but I have a question. We are looking at trying to import a blog into MOSS from either an RSS feed or XML file. Do you think your import/export changes would do the trick? Thanks Adam

Gianbattista said...

With the two hotfix kb939592 e kb939654 the versions problem is solved for both export/import and exportlist/importlist.
As you said it was a Microsoft things.

Gary Lapointe said...

Adam - Sorry for not getting back to you sooner - You could probably use the importlistitem command but it would require you to write an XSLT to transform the source data into a manifest file to match what Microsoft's code exports. I know that this is a problem I'll eventually have to solve as we have the same issue internally (we're using community server and we want to migrate all our blogs to sharepoint - I was going back and forth on whether I should make it CS specific but I like the idea of pointing to an RSS feed and using that as the source - I'll let you know if come up with anything but I suspect I won't have anything anytime soon).

Tom said...

Thanks for the extensions they worked like a charm.

Tom said...

Has anyone had issues with the copylist command or the standard import & export for a discussion board? When I use copylist or import/export upon moving to the new site the discussion board items get broken up. Meaning the discussion threads are all seperated even though in the original discussion board the threads are properly related (initial discussion topic and replies).

Gary Lapointe said...

Tom - I haven't done any testing with the discussion lists but did you try the retainobjectidentity parameter? I'm wondering if the list items have a reference to the dependent list item ID which would change during the import.

Tom said...

I did just try using the retainobjectidentity parameter after reading about it in one of your other posts. I exported my discussion board list then when attempting import using retainobjectidentity I get an error stating that my filename does not exist even though it does. Thanks for your help. I have upgraded/imported a number of sites today from sharepoint 2003 to 2007 the discussion lists are my only remaining issue.

Tom said...

The importlist command did run for me using retainobjectidentity but then I got the following error: The file Lists/General Discussion2 cannot be imported because its parent web /co
mm/msc/ does not exist.

Gary Lapointe said...

Ah yes - I forgot about that error. If you exported the file using the -nofilecompression switch you can try to hack the Manifest.xml file so that the SPWeb object identifiers match that of your new target. This could be a lot of work though (guess it depends on how many lists you need to do and how much of a concern it is to get the items migrated). There are definitely issues with the deployment API when it comes to importing lists and list items - I feel like Microsoft didn't do a whole lot of testing on this (I logged 2 bugs just yesterday and this would make three). If I have time I'll try to find a work around but to be honest I'm a bit pressed to get moving on some other items so I may not be able to get to it for a while.

Gary Lapointe said...

One more thing you can try - rather than importing the list, try importing the list items (use importlistitem along with the retainobjectidentity parameter - you'll need to export using exportlistitem). This assumes you've got an empty list to work with (you can also import the list, use deletelistitem to clear it out, and then import the items - note that everything will go to the root folder though so if you have folders they won't be preserved).

Tom said...

Thanks for the tips Gary. Your extensions have already helped out a lot. After getting the "parent web does not exist" error I did change the manifest.xml so that the all of the occurrences of the site url would match my destination url but I still didn't have success. I will also try importlistitem like you said when I get back in monday. Fortunately it's just one discussion list I need to move over.

Tom said...

Did anyone else run into issues with the import of discussion board items and come up with a solution or work around?

Gary Lapointe said...

I haven't heard anything from anyone else but that doesn't mean much. I've honestly been too swamped to try and reproduce this but I'll take a look at it as soon as I can.

Tom said...

I ended up manually posting the discussion board items using my own sharepoint account through the front end. I then modified each post by running an update statement in SQL to assign the proper author and date/time stamp.

It wasn't too bad since I had only one discussion board to import. I updated the tp_author, tp_editor, tp_modified, tp_created, and datetime1 columns within the alluserdata table.

I did notice that within the alluserdata table the tp_ThreadIndex column was null for the discussion board items I had previously imported. Which probably explains why those imported posts did not thread properly on the front end.

Gary Lapointe said...

Tom - I've just posted an updated build which fixes the issue of discussion lists not being imported correctly. The issue was due to the fact that when you import a list or list item the deployment API flattens the list (it does not honor any folder settings). The solution is to move the items back under the appropriate folders (for discussion lists the top level items are folders and and all replys are items). Unfortunately there's no way to move an item that does not have a file associated with it using the API so in order to pull this off I had to manipulate the database directly - if you intend to use these commands with discussion lists make sure you understand the ramifications towards your support when you update the database directly. The convertsubsitetositecollection, repairsitecollectionimportedfromsubsite, importlist, copylist, importlistitem, and copylistitem have all been updated to support this fix.

Agent M said...

I am sure I am just confused, but I can get the import and export to work, but the copy fails with:

"Object reference not set to an instance of an object"

Here are the commands I am using...

Export:
stsadm -o exportlist -url "http://testbi/lists/stuff2/properties.aspx" -filename c:\list1

Import:
stsadm -o importlist -url "http://testbi/model/projects/it/properties.aspx" -filename c:\list1.cmp

Copy:
stsadm -o copylist -sourceurl "http://testbi/lists/stuff2/properties.aspx" -targeturl "http://testbi/model/projects/it/properties.aspx"

The export and import get the job done, and I would figure the copy fails with error due to my syntax.

What am I doing wrong?

Gary Lapointe said...

It was a bug in my code - I had added some code to handle copying user security in the base ImportList class and forgot to test the copylist command which calls the importlist command's validation method - it was looking for the copysecuritysettings parameter which didn't exist. I fixed the code so that the parameter is added but not shown in the command help as it's not needed in the copylist command. If you download the latest it should work for you now. Thanks for finding this and letting me know about it.

Anonymous said...

I test import/export and copylist but when copy content the history isn't equal to source, example: all users of the history change for domain\system account. I need that the both document libraries (source and destiny) are same. Thanks.

Gary Lapointe said...

Did you try using the "-includeusersecurity" parameter?

Keiths said...

This is really nice extensions.

I try to run with exportlist extension, and I got 2lines of error message.
-----------------------------
Value cannot be null.
Parameter name: type
--------------------------

and following is the command line that I used.

stsadm.exe -o exportlist -url "http://MySharepointSite/Docs/Documentd/Forms/AllItems.aspx" -filename "c:\docs" -includeusersecurity -versions 4 -nofilecompression

Do you have any idea about this issue?

Thanks for you effort.

Keith

Gary Lapointe said...

That error occurs when the dll is not in the GAC.

Iris Line said...

We get this error, why? Those sites are not anomymous accessible.

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>stsa
dm -o copylist -sourceurl "http://exp.shil.irisnet.be/Lists/Services/Overview.as
px" -targeturl "http://ilt3/del/"

The Web application at http://exp.shil.irisnet.be/Lists/Services/Overview.aspx c
ould not be found. Verify that you have typed the URL correctly. If the URL shou
ld be serving existing content, the system administrator may need to add a new r
equest URL mapping to the intended application.

Gary Lapointe said...

Iris - make sure that the account you are accessing the site as has access to the web application.

David said...

Hi - these are really great extensions and something I need to use for my current task. However, I keep getting a "command line error"

All I am doing is an export off a demo system to test this out like so:

set SPAdminTool=%CommonProgramFiles%\Microsoft Shared\web server extensions\12\BIN\stsadm.exe

echo exporting list...

"%SPAdminTool%" –o exportlist -url "http://litwaredemo/Lists/Connolly%20Product%20Releases/AllItems.aspx" -filename "c:\docs\list" -includeusersecurity -versions 4 -nofilecompression

Along with the command line error stsadm dumps out a list of all the commands including yours (i.e. exportlist). I copied your DLL to c:\windows\assembly.

I would greatly appreciate any help!



TIA,
dave

Gary Lapointe said...

Becuase you're running this in a batch file you need to escape your "%" signs - %20 will be interpreted as a variable - you need to use %%20.

Anonymous said...

Having used Import to move a task list from one domain to another, I noticed all the assigned to field are blank is there a clever way to reassign them. Users do have accounts in both domains.?

Anonymous said...

Regarding the versioning problem stated by Gianbattiston the 11th October 2007, i.e.

file1 whith
v2 By Author "A" date "dateX"
v1 By Author "B" data "DateY"
after export/import in the new list become:
v2 By Author "A" date "DateX"
v1 By Author "A" date "DateX"

is also happening to me. I have MOSS2007 with SP1.

Does anyone know why?

Anonymous said...

When I import a list to a site which hasn`t got the content type being used in the list it copies de content type.This is very nice , but the problem is that I can`t see the content type in the 'Site content types' gallery but I can see the content type in the imported library.
Does anyone know how to make the contenttype visible in the gallery?
I have the same problem with Site Columns.

Gary Lapointe said...

This is actually a limitation of the deployment API. Content types can be site collection scoped, web scoped, or list scoped. When you bind a content type to a list it will always create a list scoped content type that inherits from the web or site collection content type. To get around this you should be able to export the content type (use exportcontenttypes) and deploy it using a feature and then import your list - this "should" preserve the inheritence structure.

Justin French said...

Hi Gary,

Is it possible to use these commands to incrementally update a list? For example, for the first time i import, I retain the id's. Then if i do further export/imports, will only new/modified versions be imported?

Basically, I'm thinking of using these commands to migrate list content from a content entry environment to a live environment, but don't want to delete/recreate each time.

Thanks,

Justin

Gary Lapointe said...

Justin - yes, you can do that but you'll probably want to work with the exportlistitem and importlistitem commands instead - I'd recommend playing with it in a test environment first though and try both sets of commands to see what works for you as there are many potential issues with the deployment API which these commands use (the updateversions and retainobjectidentity parameters will be the ones you'll want to play around with).

Obieg Dokumentow said...

On Codeplex I saw a custom workflow activity which copies/moves documents from one library to another along with version history. Could this approach be used here?

Doc said...

Hi!
I used the following command:
stsadm –o importlist -url "http://intranet/xxx/" -filename "c:\temp\test\dv.cmp" -includeusersecurity -updateversions 2 -nofilecompression -retargetlinks -sourceurl "http://intranet/einrichtungen/zentralverwaltung/dienstanweisungenvereinbarungen/Forms/AllItems.aspx"

it produces an error like this:

Fehler in der Befehlszeile.
it can be translated into something like:
error/failure in the command line

any suggestions?
thanks chris

Adam Toth said...

Hi Gary,

I get the same error as Iris. I am running the tool on a dev server, trying to copy a list from the production server over to the dev server.

The account I am running under is a local admin, farm admin, and site collection admin on both servers. The servers are both 64bit.

Is there anything else I need?

Adam Toth said...

Hi Gary,

I also had a problem with exporting a list from one server and then importing to another. I got the following error during import. This was right after it finished importing the users and groups:

The element 'Fields' in namespace 'urn:deployment-manifest-schema' has invalid child element 'Field' in namespace 'http://schemas.microsoft.com/sharepoint/soap/'. List of possible elements expected: 'FieldRef, Field' in namespace 'urn:deployment-manifest-schema'.

Any ideas?

Gary Lapointe said...

Doc - the only time I've seen this occur (assuming the parameters are all correct which I believe they are) is when someone has copied and pasted the text to the command window and some of the pasted text retained invalid encodings (such as the "-o" parameter in your comment) - my suggestion is to retype the command manually. Hope this helps.

Gary Lapointe said...

Adam - if you're trying to use copylist to copy a list from one farm to another it won't work - the copylist command only works within a single farm. What you need to do is use exportlist on your source farm and importlist on your target (copylist just wraps these two commands up).

Gary Lapointe said...

Adam - on this other error you are getting regarding the Field on the import I'm a bit stumped - it's saying that Field is invalid and then says that valid entries are FieldRef and Field - my only guess is that maybe the encoding of the files got hosed up somehow???

Gary Lapointe said...

Obieg - sorry for not getting back to you sooner (comment got lost in my email) - I haven't looked at the codeplex project you mention but to answer your question I'd say that yes, you could probably abstract the code from my commands and wrap them in such a way that you could use them in a workflow activity or event receiver (I believe that's what you are asking?).

MikeG said...

Hi Gary, Thanks for a really amazing set of tools.

What advice would you have about moving around doc libaries and preserving versioning?

Do any of the tools enable this?

Mike

Gary Lapointe said...

The exportlist and importlist commands will enable you to move a doc library around and has options to control how versioning is handled.

MikeG said...

Hi Gary,
When I try to run the copylist command, I get the following error:

Completed with 1 errors.



Log file generated:

C:\Documents and Settings\gallagherM.eu\Local Settings\Temp\86c503c8-aaf

b-4d5d-8d35-fbbb79348873\export.log

Violation of PRIMARY KEY constraint 'PK__#ExportObjects____0E8B5888'. Cannot ins

ert duplicate key in object 'dbo.#ExportObjects'.

The statement has been terminated.



Any ideas?

Thanks!
Mike

Gary Lapointe said...

Mike- I've seen the error before but I don't have a fix for it - it's an issue with the MS deployment API. You might search around on the sharepoint forums on MSDN as I believe this error has been discussed there (can't remember if anyone had a solution though).

Anonymous said...

Hi Gary, i also thanks for this amazing set of tools.

Generally, the copy function runs fine. If i have a List with a InfoPath content-type form, a workflowdefinition and an InfoPath-Doc in the list, then i get the following error:

FatalError: Object reference not set to an instance of an object.

Here are the command I am using...
stsadm -o gl-copylist -sourceurl "http://dev.moss/sv/ittk/FormCopyTest/Forms/AllItems.aspx" -targeturl "http://dev.moss/sv/" -updateversions 2 -versions 4 -retargetlinks

If i have no InfoPath-Doc in the list, the copy runs.

I have MOSS2007 with SP1 and Office 2007 (InfoPath) with SP1.

Any Details on the log shown here (i hope, its not to much):
....
Progress: Importing File FormCopyTest/Forms/template.xml.
Progress: Importing File FormCopyTest/Forms/Upload.aspx.
Progress: Importing File FormCopyTest/Forms/WebFldr.aspx.
FatalError: Object reference not set to an instance of an object.
at Microsoft.SharePoint.Deployment.ContentTypeSerializer.GetContentType(SPContentType sourceContentType, ImportObjectManager importObjectManager, Boolean isParentSystemObject)
at Microsoft.SharePoint.Deployment.ContentTypeSerializer.ProcessContentType(SPContentType sourceContentType, String contentTypeXml, ImportObjectManager importObjectManager, Boolean IsParentSystemObject)
at Microsoft.SharePoint.Deployment.ContentTypeSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 4/23/2008 8:05:32 AM.
Completed with 2 warnings.
Completed with 1 errors.
...

Do you have any idea about this issue?

Thanks for you effort
Thomas

Brendan said...

Gary

Love the commands. I'm getting an error when I try to copy. I saw someone had already mentioned it but I'm still getting the error. I just downloaded the extensions today so I'm assuming it is the latest version.

I get the "list item cannot be imported because its parent web does not exist" error on a copy. The export goes well but the import does not. Any ideas?

Garven said...

Hi- I have used the export and import to move sites between machines. The only thing that i seem not to find is the temporary folder used for the compressing and the extracting of the files. My target machine does not have a big C: drive partition so I keep on getting an error on insufficient disk space. How can I change this?

Anonymous said...

This works like charm and is a very useful tool for us. The one issue I foresee is that as this runs, a temporary directory to store data is created here...

C:\Documents and Settings\someuser\Local Settings\Temp

On our server builds we partition them in such a way that ONLY the OS lives on the c:\ partition. Typically this runs with perhaps 2-3 Gb. of free space. Suppose I need to migrate 30 Gb. of data. Does that temp directory flush itself as it runs out of space or does stsadm just puke?

Anbody run into this... have a solution?

Gary Lapointe said...

Thomas - unfortunately this is a bug with the deployment API and I have no work around for it. I believe that Microsoft is working on a post service pack rollup for some of the deployment API issues but I have no timeline information for it and don't have any details on what they are planning on fixing.

Gary Lapointe said...

Brendan - this error usually occurs when you are trying to reparent the list (most likely to get when using the retainobjectidentity flag with the import command). Not sure why you'd get it if you were using the copylist command as it doesn't retain identifiers but I do know that there are a lot of bugs with the deployment API - I'd try using the exportlist and importlist commands individually and try some of the different options but in the end we're at the mercy of the deployment API.

Gary Lapointe said...

Garven - I've added a -temppath parameter to the copylist command which you can use to specify the directory to store the files.

Tanzim Akhtar said...

I tried to run the following command given in your example for gl-copylist:

stsadm –o gl-copylist -sourceurl "http://intranet/Documents/Forms/AllItems.aspx" -targeturl "http://teamsites/" -includeusersecurity -updateversions 2 -versions 4 - nofilecompression -retargetlinks

I received the following error:

Command line error. Invalid parameter: nofilecompression.

I think the option nofilecompression is not there in gl-copylist.

Thank you.

Gary Lapointe said...

Tanzim - that's my bad - I added that parameter into my example by accident - the syntax shown in the console window is correct. I'll update the post shortly.

Adam Toth said...

Hi Gary,

I finally got around to doing some investigation about the error I posted earlier regarding Field not being a valid child element of Fields.

It turns out this was an issue with Tzunami, a migration product. The product was used to import documents from Documentum into a document library in Sharepoint. I was then trying to take that library and copy it to a different location in the site collection.

When Tzunami imported the documents with metadata, it created additional columns on the library, and when it created these fields, it screwed them up somehow and added an xmlns attribute to the field.

I cracked open the manifest file and noticed that the Tzunami columns all had an xmlns="http://schemas.microsoft.com/sharepoint/soap/" attribute on the Field element, but none of the other columns had that. I removed that attribute and everything ran again.

Thanks, it worked great.

Adam Toth said...

Hi Gary,

Two questions:

1. Is it possible to copy a list to the same web as the original list, but with just a different list name?

2. When I export and import a list that has multiple content types allowed, after import, all the documents are set to the incorrect content type. I've got a list with two content types (Document and a custom one), and all docs are set to the custom content type. After export and import, the new list has all the documents set to the "Document" type. I tried again changing the default content type to be the custom content type, but no luck. Do you know why that might be?

Gary Lapointe said...

Adam - unfortunately both of your questions come down to limitations and bugs with the content deployment API. I know MSFT is working on fixing to the API but I have no timeline as to when that will be available.

Anonymous said...

Great work here, I ran into an issue when exporting and then importing a discussion board. The error is below. Any idea?

FatalError: Error in the application.
at Microsoft.SharePoint.SPList.UpdateDirectoryManagementService(String oldAlias, String newAlias)
at Microsoft.SharePoint.SPList.Update(Boolean bFromMigration)
at Microsoft.SharePoint.Deployment.ListSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()

Thanks,

Malik

Gary Lapointe said...

Unfortunately I don't have any suggestions - the deployment API is riddled with bugs and this appears to be yet one more of them.

Anonymous said...

I got an Object Reference error on addsolution and it was an access denied error
http://techdhaan.wordpress.com/2008/05/18/stsadm-addsolution-and-deploysolution-object-reference-not-set-to-an-instance-of-the-object-error/

Anonymous said...

The export seems to work fine but when I run the import, it fails with the error "Requirements.xml was not found no verifications ran". There is definitely no requirements.xml file that it is looking for. I'm using both -includeusersecurity and -nofilecompression commands during export and import. Thanks for any advice.

Gary Lapointe said...

The Requirements.xml file may exist via link or may have once existed. There's lots of issues with dependencies and what not. Try setting the -includedependencies parameter to "none" or "content" and see if that helps. Failing that try to get the content deployment QFE and see if that helps. Failing that, you'll need to dig through your site to see where the reference to the xml file is and get rid of it.

turtlehand said...

Gary,
Thanks for all your work on these extensions.

I'm having trouble copying a list; when I run:

stsadm -o gl-copylist
-sourceurl "http://noblepoint/IT/Lists/Infrastucture Tasks/AllItems.aspx"
-targeturl "http://noblepoint/IT/Lists/Initiative Tracking/"
-versions 4
-temppath C:\Temp\Exports\
-haltonfatalerror
-updateversions 2

The commandline output shows that it exports and imports fine. 0 warnings and 0 errors but when nothing appears in the destination list. What am I doing wrong?

Anonymous said...

Gary,

Impressive set of tools.

I'm trying to export/import a list between MOSS2007 AND WSS3.0.
The export works like a treat but the import is failing with the following:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>stsadm -o gl-importlist
-url "http://www.exonynet.com/development/shared documents/forms/allItems.aspx" -filename "d:\forcel
ist.cmp" -retargetlinks -sourceurl "http://xshare/developement/force/lists/force cases/allitems.aspx
"

Start Time: 9/25/2008 2:06:41 PM.
Progress: Initializing Import.
FatalError: The specified user EXONYSHARE\peter could not be found.
at Microsoft.SharePoint.Deployment.DeploymentStreamingContext..ctor(ObjectManager objectManager,
DataFileManager fileManager, ViewFormsList viewForms)
at Microsoft.SharePoint.Deployment.ImportStreamingContext..ctor(ImportObjectManager objectManager
, ImportDataFileManager importFileManager, SPImportSettings importSettings, ViewFormsList viewList)
at Microsoft.SharePoint.Deployment.ImportObjectManager..ctor(SPDeployment deploymentDriver, SPSit
e site, SPImportSettings importSettings, ImportDataFileManager dataFileManager, ViewFormsList viewFo
rms)
at Microsoft.SharePoint.Deployment.SPImport.InitializeImport()
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 9/25/2008 2:06:41 PM.
Completed with 0 warnings.
Completed with 1 errors.

Log file generated:
d:\forcelist.cmp.import.log

The specified user EXONYSHARE\peter could not be found.

The WSS is on a hosted environment in a workgroup. All users are part of the fbamembers but non of the local users (EXONYSHARE) appear in the users list.

Any ideas?
Thanks P.

Peter Belton said...

Gary,

Thanks for the extensions, I've made great use of them migrating from Sharepoint Server 3.0 to WSS 3.0.

I've run into a problem on exporting our document library I'm getting a object reference not set error.

Here's the commands I'm using and the error.
stsadm -o gl-exportlist -url "http://xshare/docs/documents/forms/allitems.aspx" -filename c:\docs\docs -includedescendants All

stsadm -o gl-importlist -url "http://www.exonynet.com/docs/Documents1/forms/allitems.aspx" -filename "d:\migration\docs\docs.cmp"




Start Time: 9/29/2008 8:39:30 AM.
Progress: Initializing Import.
Progress: Starting content import.
Progress: De-Serializing Objects to Database.
Progress: Importing Folder /Development/Projects/SitUp MultiVariant Phase 1/Forms/Document.
Progress: Importing Folder /docs/Documents.
Progress: Importing Folder /docs/Documents/Forms.
Progress: Importing Folder /docs/Documents/Forms/Document.
Progress: Importing Folder /KnowledgeBase/SRP/Wiki Pages/Forms/Wiki Page(-1096469639).
Progress: Importing File Documents/Forms/AllItems.aspx.
Progress: Importing File Documents/Forms/Combine.aspx.
Progress: Importing File Documents/Forms/DispForm.aspx.
Progress: Importing File Documents/Forms/EditForm.aspx.
Progress: Importing File Documents/Forms/PersonalViews.aspx.
Progress: Importing File Documents/Forms/repair.aspx.
Progress: Importing File Documents/Forms/template.doc.
Progress: Importing File Documents/Forms/Upload.aspx.
Progress: Importing File Documents/Forms/WebFldr.aspx.
FatalError: Object reference not set to an instance of an object.
at Microsoft.SharePoint.Deployment.SecurityCheck.OnContentTypeImport(DeploymentStreamingContext context, SPList list)

at Microsoft.SharePoint.Deployment.ContentTypeSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingC
ontext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentO
bject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 9/29/2008 8:44:21 AM.
Completed with 0 warnings.
Completed with 1 errors.

Gary Lapointe said...

Peter - for the import try setting the includedescendents to none. The import is trying to bring in loosely related items and in this case it's trying to bring in a content type associated with a list that does not exist (this is Microsoft's code that does this - I don't really have any control over it).

sos said...

Gary I have already successfully exported imported site olumns and content types using your helpful I'm getting "command line error when I try to run following command . stsadm –o gl-exportlist -url "http://intranet.net/Lists/My%20forms/AllItems.aspx" -filename "c:\docs" -includeusersecurity -versions 4 -nofilecompression

Gary Lapointe said...

Try retyping the command in by hand (don't copy and paste - looks like your -o has a special character in it).

Anonymous said...

First, let me say thanks so much for making these extensions available. They're great, and very much needed.

Now, on to my problem. I'm trying to use the gl-copylist command, and I'm receiving this error: FatalError: Value does not fall within the expected range.
at Microsoft.SharePoint.SPWeb.GetWebRelativeUrlFromUrl(String strUrl, Boolean
includeQueryString, Boolean canonicalizeUrl)

Any suggestions? Is there any chance this is related to the fact that I'm on a government intranet that uses a custom top-level domain?

Thanks for any assistance.

Gary Lapointe said...

Honestly I'm not sure what the deal is with that one - is there a more complete stack trace that you can share (might help to narrow the problem down)?

Anonymous said...

Hi Gary,

I left you a question somewhere on your blog and now cannot find my way back to it. (sheepish grin).

I am having a problem with the gl-exportlist command. The command starts but then I see the message " Warning: A web part or web form control type could not be found or is not registered as safe. The web part will still be exported."

But when I try to import the list to another site, I get "Warning: could not find webpart [Unknown:guid number] in the safe controls list. I am using the farm level account for this.

Any idea what is wrong? The other extensions that I have used seem to be fine.

Thanks.

Kate

Gary Lapointe said...

Kate - There's a good chance that you have a reference to a web part that exists only in the bin folder and/or is not registered as safe. I've seen situations where I'd get this error if the dll for the web part wasn't in the GAC. The warning is just a precursor to the error you'll get on the import (it still exports the web part to the extent that it can). The trick for you will be to narrow down what specific web part is causing you the issue and then make sure that it is registered as safe in both your source and target sites (web.config safecontrols section) and possibly (temporarily) add the web part to the GAC. It's a hack I know but it's the only way I've been able to get around the issue when I got it.

Sigvaldi Óskar said...

Hi Gary,

We have used your gl-exporlist to exprt custom lists but when we try to import using gl-importlist we get an error. We are doing this between different web applications. The export command line is

stsadm -o gl-exportlist -url "http://host1/Lists/Verkbeinir/Allar%20verkbeinir.aspx" -filename "z:\listexport\verkbeidnir" -includeusersecurity -nofilecompression

This seems to work fine without errors. The import command line is

stsadm -o gl-importlist -url "http://host2/vinnubord/UT" -filename "z:\listexport\verkbeidnir" -includeusersecurity -nofilecompression

The import creates the custom list, installs the views and creates folders for attachements but then returns with error (and the created list is empty):

...
[10/15/2008 11:03:50 AM]: Progress: Importing List Verkbeiðnir.
[10/15/2008 11:03:55 AM]: FatalError: Could not load file or assembly 'SharepointListLibHandler\, Version\=1.0.0.0\, Culture\=neutral\, PublicKeyToken\={e73cdd5c0ad48b16}' or one of its dependencies. The given assembly name or codebase was invalid. (Exception from HRESULT: 0x80131047)
at Microsoft.SharePoint.SPEventReceiverDefinition.ValidAssembly()
at Microsoft.SharePoint.SPEventReceiverDefinition.ValidReceiverFields()
at Microsoft.SharePoint.SPEventReceiverDefinition.UpdateInternal(Boolean isMigration)
at Microsoft.SharePoint.Deployment.EventReceiverSerializer.UpdateEventReceiver(SPEventReceiverDefinition eventReceiver, XmlElement eventReceiverData, SPImportSettings settings)
at Microsoft.SharePoint.Deployment.EventReceiverSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.ListSerializer.UpdateListEventReceivers(SPList list, Dictionary`2 listMetaData, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.ListSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()
[10/15/2008 11:03:56 AM]: Progress: Import Completed.
[10/15/2008 11:03:56 AM]: Finish Time: 10/15/2008 11:03:56 AM.
[10/15/2008 11:03:56 AM]: Completed with 0 warnings.
[10/15/2008 11:03:56 AM]: Completed with 1 errors.

Any idea what is the reason for this error?

Gary Lapointe said...

Looks like you had a custom event handler that was deployed (probably as a feature) to your source host and not your target - make sure that if the dll that the receiver belongs to is in the app_bin folder or the GAC.

Brian said...

There is a SharePoint bug that I was hoping you had worked around, but apparently not. If you have a calculated field made up of other fields and those fields have a space in their name, the import fails.

AlexMon said...

I am a little fuzzy on how the -includedescendants switch works with gl-exportlist. Is there any combination of commands that will, for example, copy both task list items and their related rows in a foreign workflow list?

Anonymous said...

Thank you, it's very useful

Gordon Hickley said...

Some excellent tools here thank you.

I would just like to suggest for the copy operations that they check the target URL is valid before performing the export step. Several times I have watched the successful export side work only for the copy to then immeidately terminate because I made a typo with the target URL.

Toshi said...

This is a set of really useful extensions. I'm still rummaging through the whole command list since I've just found out the whole thing last week.

I did a gl-copylist on an old subsite that i want to move to a new subsite. Found some anomalies though that might perhaps be solved with another extension.

1 - Columns in the view that were used to denote workflow status came across as custom columns and had to be remove manually

2 - One of the columns were meant to be hidden but for some reason is now shown as free-form field..

The command I've used are..

stsadm -o gl-copylist -sourceurl "%sourceurl%/DocLib/Forms/AllItems.aspx" -targeturl "%targeturl%/" -includeusersecurity -updateversions 2 -versions 4 -retargetlinks

Gary Lapointe said...

Unfortunately these commands are subject to the limitations of the content deployment API (I'm just wraping that API). You might want to make sure that you have the infrastructure update installed - there were numerous fixes to the content deployment API in the IU.

Ruben said...

Hi,

Great tools Gary.

I have one question. i Tried to use gl-copylist tool and the export part went well. but when it starts importing list items I get a fatal error:
Invalid template URL.

can anyone help me please!

OferGal said...

I used copy with:
stsadm –o gl-copylist -sourceurl "http://litwaredemo/salesmarketing/Marketing Document Library/Forms/Customer Ready Versions.aspx" -targeturl "http://sprmoss/MPS/" -versions 4 -retargetlinks

but I get Command line error.
I did before the 3 statesments to install the Lapointe.SharePoint.STSADM.Commands.wsp file

Is there something wrong with the sysntax?

Gary Lapointe said...

It's the dashes - you need to retype them (you probably copied from my blog - stupid Live Writer was converting them to a special character dash and I didn't catch it until way late).

Anonymous said...

See http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.deployment.spexportobject.includedescendants.aspx for includedescendants spec.

Anonymous said...

(Riza) Copy Contacts List - If the contacts have the same last name the permissions for the list items are not copied correctly, if you give unique persmissions for each contact;
Smith, George - Group A, User X
Smith, Sara - Group B, User Z

after copy operation each smith doesn't have the their original permission settings. I tried to find a solution for this problem, basically I haven't figure it out any solution for this problem(I am not using retain object ID) yet. You are using list item name to identify a list item but it is not valid for some lists like Contacts List or is there a other to work around this issue? - Riza

Anonymous said...

Thanks for an excellent work.

TA said...

Amazing work here and its got my 99% percent there. Quick question though, The list export finishes with 4 warnings (webpart not registered as safe)
We are trying to take the list from a customized deployement to out of the box wss install. When I continue with the import everything seems to run fine but whenever I click on a item to view it or click on "New" it gives me the AWSOME wss error "an error has occured" and thats it. Any ideas or is it the obvious answer?

Gary Lapointe said...

I guess I'd have to start with the obvious - find out which web parts it's complaining about and make sure there's a safe controls entry.

Durga said...

I am getting the following import error during list import (gl-importlist). Any help would be apreciated.

Progress: Importing File Lists/bwcal1/my-sub.aspx.
Progress: Importing File Lists/bwcal1/NewForm.aspx.
FatalError: The element 'Fields' in namespace 'urn:deployment-manifest-schema' h
as invalid child element 'Field' in namespace 'http://schemas.microsoft.com/shar
epoint/'. List of possible elements expected: 'FieldRef, Field' in namespace 'ur
n:deployment-manifest-schema'.
at System.Xml.Schema.XmlSchemaValidator.SendValidationEvent(ValidationEventHa
ndler eventHandler, Object sender, XmlSchemaValidationException e, XmlSeverityTy
pe severity)
at System.Xml.Schema.XmlSchemaValidator.ElementValidationError(XmlQualifiedNa
me name, ValidationState context, ValidationEventHandler eventHandler, Object se
nder, String sourceUri, Int32 lineNo, Int32 linePos, Boolean getParticles)
at System.Xml.Schema.XmlSchemaValidator.ValidateElementContext(XmlQualifiedNa
me elementName, Boolean& invalidElementInContext)
at System.Xml.Schema.XmlSchemaValidator.ValidateElement(String localName, Str
ing namespaceUri, XmlSchemaInfo schemaInfo, String xsiType, String xsiNil, Strin
g xsiSchemaLocation, String xsiNoNamespaceSchemaLocation)
at System.Xml.XsdValidatingReader.ProcessElementEvent()
at System.Xml.XsdValidatingReader.ProcessReaderEvent()
at System.Xml.XsdValidatingReader.Read()
at System.Xml.XmlWriter.WriteNode(XmlReader reader, Boolean defattr)
at System.Xml.XmlReader.ReadOuterXml()
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType,
Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type object
Type, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializat
ionStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serial
izationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReade
r xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 5/5/2009 1:04:56 PM.
Completed with 0 warnings.
Completed with 1 errors.

Log file generated:
C:\temp2\ExportTest\d_brazilbwcal1aftprop.cab.import.log

The element 'Fields' in namespace 'urn:deployment-manifest-schema' has invalid c
hild element 'Field' in namespace 'http://schemas.microsoft.com/sharepoint/'. Li
st of possible elements expected: 'FieldRef, Field' in namespace 'urn:deployment
-manifest-schema'.

Anonymous said...

Hi Gary,

I think the work you do is absolutely great!
My problem is the following (forgive me if this has already been covered but there are soooo many comments):
copying a document library from one site to another works great but when I use the TOC Web Part the doc libs do not show up. If I create the doc lib manually it shows up in the TOC. As soon as I use the copylist function the name of the previously added doc lib disappears (the copied list has the same name).
Any response would be great!

Best regards

Uli Netzer

West said...

Hi,

I am trying to copy a discussion board from one web app to another. I am using gl-copylist. The problem I'm having is it "does not" overwrite the list specified in the "targeturl" parameter. It does copy the list, but it doesn't overwrite the the list specified in the -targeturl which is what I want. So in short, I end up with 2 lists.

Any idea?

-Westside

Gary Lapointe said...

Haven't looked at the command in a while but I believe you just need to make sure the URL is the same. Otherwise use the gl-copylistitems command.

Gary Lapointe said...

Uli - I'm not sure why it's not showing up in the TOC web part - I'm currently on vaca so I'm not anywhere near a SP box to play with this - first thing I'd check is permissions though I doubt that's the issue.

T Stan said...

I hope you are enjoying your vacation, Gary. Your STSADM updates kick major butt. I'm a big fan. The import has worked for me very well until just today. I was wondering if you could offer any suggestions as to why a list gets the following error.

Command:
stsadm -o gl-importlist -url "http://server/newsite" -filename "list.bak" -includeusersecurity -updateversions 2 -nofilecompression -retargetlinks -sourceurl "http://server/oldsite/oldlist"

Error:
[6/12/2009 11:07:27 AM]: Progress: Importing File Project Charters/Forms/AllItems.aspx.
[6/12/2009 11:07:27 AM]: FatalError: Value does not fall within the expected range.
at Microsoft.SharePoint.SPWeb.GetWebRelativeUrlFromUrl(String strUrl, Boolean includeQueryString, Boolean canonicalizeUrl)
at Microsoft.SharePoint.SPWeb.GetWebRelativeUrlFromUrl(String strUrl)
at Microsoft.SharePoint.Deployment.FileSerializer.GetFile(SPWeb parentWeb, Guid fileId, String fileUrl, Guid parentId, Guid listId, Boolean& isDeleted, Boolean& fileExistsInDb, String& destFileUrl)
at Microsoft.SharePoint.Deployment.FileSerializer.GetFile(SPWeb parentWeb, SerializationInfoHelper infoHelper, Boolean& isDeleted, Boolean& fileExistsInDb, String& destFileUrl)
at Microsoft.SharePoint.Deployment.FileSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()
[6/12/2009 11:07:27 AM]: Progress: Import Completed.
[6/12/2009 11:07:27 AM]: Finish Time: 6/12/2009 11:07:27 AM.
[6/12/2009 11:07:27 AM]: Completed with 0 warnings.
[6/12/2009 11:07:27 AM]: Completed with 1 errors.

Gary Lapointe said...

Vacation is great - just gotta learn to keep off my email :)

Try passing in the -excludedependencies parameter.

T Stan said...

Not only are your tools great, but your tech support rocks, too!

Adding the switch "-excludedependencies" made my error on import go away. I had read your suggestion to use that switch in the initial post, but I guess I thought I would see an error right away, rather than at the end of the process. I thought wrong.

Thank you very much,

- T Stan

Frank-Ove Kristiansen said...

Hi Gary,

I've tested several of your commands, including gl-moveweb, gl-exportlist, gl-copylist, gl-exportlistfield and gl-updatelistfield.

I have a site collection that I need to move to be a subsite in another site collection. In this site collection, I have some lookup fields in my lists. The lookup fields gets data from lists in the site, obviously.

Here's my problem: no matter what I try using the built-in commands or your commands, the lookup fields get screwed up. I.e. the lookup field doesn't seem to link to the source list, even though I've copied that first (this is related to the gl-copylist testing I did).

Do you have any clue or tips regarding this?

Thank you for any help, and enjoy your vacation! :-)

Regards,
Frank

Gary Lapointe said...

Frank, the trick is that you need to be able to maintain the object identities of the lists and webs - this is what the retainobjectidentity parameter is for - problem is that you are trying to move a site collection to a sub-site so you're likely to end up with all kinds of issues in trying to use the retainobjectidentity parameter. You can try to move the lists over individually by hacking the manifest.xml file that is exported and manipulating the web IDs to match that of your target webs but it will be painful. Another approach will be to write some code to re-wire up the lookups after the import.

Per Bjarte Sjømæling said...

Hello, thank you for your great stsadm extensions, I'm using them a lot!

Here my problem: A customer wants to restore a document library on a site, the library had become "screwed up". There's a bunch of libraries in the site, and the customer wants to restore just one of them. The library contains very important information (documents and their metadata). The library uses both major and minor versioning. The history of the documents are also important, so they want/need to restore all versions. Customer runs MOSS with SP2 and last versions of stsadm extensions.

My plan for the restore operation: First delete the production library, then a) restore the site collection to a test server, b) do a gl-exportlist with - versions 4 to export everything in the library and lastly c) do a gl-importlist and then gl-import

a) and b) went fine, so did c) - but the next day we found a problem: Just the last major version and its subsequent minor versions have been imported. This is also confirmed by comparing the export and import logs.

Question: Have I missed something, should it be possible to import all versions, both majors and minors? Or is this a limitation in the underlying API? (I've tried - just in case - to use the -updateversions 2 parameter, but this doesn't help. Just now I'm testing using gl-copylist, it seems this gives the same result.)

Best regards
Per Bjarte

Per Bjarte Sjømæling said...

I've just posted a comment regarding gl-importlist - and indications that just last major and its subsequent minors will be imported. It seems I have to do some further investigation and come back on this. Definitely not all versions are not imported. The site and the library has been migrated from SPS2003.

Ann said...

First of all,thank you so much for these commands. These are awesome!
Is there a limitation on what size of a list one can export/import using these commands? I have a 22Gb list I need to move. Will these commands work for it? Thanks in advance.

Anonymous said...

Gary,Thanks for the commands and more importantly...thanks for sharing them.
I am using the gl-exportlist/importlist commands to move a document library from an existing subsite to a new site collection. Everything seems to have been migrated fine. However the user security permissions have not been copied over. I used the [-includeusersecurity] switch as well as the [-copysecuritysettings] switch but it did not help. Am I missing something?

Gary Lapointe said...

Were there any errors? Try running the gl-exportlistsecurity and gl-importlistsecurity and see if you get any additional information.

Jay Pollack said...

This is a great tool! I've used both the import/exportlist commands as well as the "publishlistitems" one and they've been a great help (the publishlistitem one saved us HOURS of manually checking in docs!)

I need to go look at what the rest of the commands can do - but these are amazing. THANK YOU!!

Pat Medlock said...

I have just some standalone ASPX pages in my site. They are not in a document library, just created with SP Designer. Is there any tool that I can use just to move single pages?

Gary Lapointe said...

It can be done using the object model but I'm not aware of anything that will operate on files outside of a library.

Sweet Sparky said...

Hi Gary,
Back on your excellent tools. I thought it would have been great if somehow we maintain the same id(guid) for the list at that target. This way one does not have to sync all the web parts. any thoughts.
MC

Gary Lapointe said...

The retainobjectidentity parameter will do that for you but there are caveats around when you can use it.

Thomas said...

Hey Gary,
first, thanks a lot for this great tool :-)

I have a problem with gl-copylist. I have copied a Tasklist to a Subwebsite, but the "assign to" is empty in the copied list.
Is this a bug or is it my mistaken?

Thanks for helping me.

Gary Lapointe said...

Did you try including the -includeusersecurity parameter? Tasks lists are a bit funny so there could be something odd with the list itself. Also make sure you are running at least SP2.

Thomas said...

Hey Gary,

it was a problem with the missing -includeusersecurity parameter.
Now it's working fine.

Thanks for your help :-)

Stephen said...

It works perfect for me! Thank you for doing the good job.

mnateras said...

Hi Gary,

I've been using you tool quiet a lot for moving lists between servers using the gl-importlist and gl-exportlist, seems like there are sometimes when I have issues moving the attachments, like this this time, if you can, please take a look at the error I get this once I try to import the files generated by gl-exportlist:

Start Time: 9/30/2009 4:51:08 PM.
Progress: Initializing Import.
Warning: Import requirement file C:\Users\mnateras\AppData\Local\Temp\1\112ab70c
-0a4b-4ea1-9556-87b0dfa48dc0\Requirements.xml was not found no verifications ran
.
FatalError: Could not find file 'C:\Users\mnateras\AppData\Local\Temp\1\112ab70c
-0a4b-4ea1-9556-87b0dfa48dc0\ExportSettings.xml'.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, I
nt32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions o
ptions, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access,
FileShare share)
at System.IO.FileInfo.Open(FileMode mode, FileAccess access)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeExportSettings()
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 9/30/2009 4:51:09 PM.
Completed with 1 warnings.
Completed with 1 errors.

Log file generated:
C:\Users\mnateras\Documents\Visual Studio 2008\Projects\....

I already try to look for that file manually but is not there anymore

Thanks for your time

Gary Lapointe said...

Not sure what's going on - did you try exporting to a different location?

Anagha said...

Gary, thanks a lot for these. I have used them to move several lists and they work great.
However I get an error for one particular list I am tring to move to a different site collection (The list is over 14GB) The error is:

Progress: Calculating Objects to Export.
FatalError: String or binary data would be truncated.
The statement has been terminated.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolea
n breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObj
ect stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cm
dHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, Tds
ParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, Run
Behavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBe
havior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehav
ior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult
result)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult res
ult, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.SharePoint.Utilities.SqlSession.ExecuteNonQuery(SqlCommand comma
nd)
at Microsoft.SharePoint.Deployment.SPExport.CalculateChildrenToExport(Boolean
linksOnly)
at Microsoft.SharePoint.Deployment.SPExport.CalculateObjectsToExport()
at Microsoft.SharePoint.Deployment.SPExport.Run()
Progress: Export Completed.
Finish Time: 10/12/2009 2:25:52 PM.
Completed with 0 warnings.
Completed with 1 errors.

I am using the command:

stsadm-o gl-exportlist -url "URL" -filename "FILENAME" -overwrite -includeusersecurity -versions 4 -cabsize 900 -nofilecompression

Any help will be appreciated. Thanks

franzey said...

What is required for the -includeusersecurity to work? I've tried multiple times to copy a document library from one site to another site and it has worked each time, EXCEPT for the security is automatically inherited from the site. The document level permissions are not copied over. Any ideas?

Anonymous said...

Just want to say a big thanks, It works perfect great job.

Stan Spotts said...

I'm hoping it's just user error, but I can't seem to get this to export a list from an SSL secured URL. I tried it with a local SharePoint instance (http://) and it worked fine, but when I tried it with our production site (https://) it told me:

The Web application at https://intranet.company.com/Backlog/Backlog/AllItems.aspx could not be found. Verify that you have typed the URL correctly. If the URL should be serving existing content, the system administrator may need to add a new request URL mapping to the intended application.

Gary Lapointe said...

Make sure you're on the same production server when you run the command. Make sure you use the default AAM for the web app.

SchuhDi said...

I am using the gl-exportlist and gl-importlist extensions a lot. Overall it works very well.
But in some cases I receive an error like this


[12/7/2009 12:17:38 PM]: FatalError: Unexpected XML declaration. The XML declaration must be the first node in the document, and no white space characters are allowed to appear before it. Line 251, position 532.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ParsePI(BufferBuilder piInDtdStringBuilder)
at System.Xml.XmlTextReaderImpl.ParseElementContent()
at System.Xml.XsdValidatingReader.Read()
at System.Xml.XmlWriter.WriteNode(XmlReader reader, Boolean defattr)
at System.Xml.XmlReader.ReadOuterXml()
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()



It seems, that in this case the inner xml of the listFormWebPart in the manifest.xml is inavlid. In other(successful) cases there is no inner xml for this web part in the manifest.xml.

Any ideas?

Rohit said...

From Where i can download this commands. As this command is not available in STSADM. So i need to install it before i can use it. Can u plz help me

Gary Lapointe said...

You can download the the WSP from the downloads page.

Anonymous said...

I have a couple of people picker fields that are being imported as blank. Is there a setting for this?

akorelc said...

Gary - just have to tell you that the export/import worked like a charm when I had to recover data and move it from one server to another. Thanks a bunch!

MC said...

Hi Gary,
I am having the same issue faced with Turtle hand posting. I did not see a resolution and hence posting this again.
I had to make a duplicate of a list with the same site. Hence I opted for the exportlist first and then used the importlist. The Log is saying that it was imported successfully with 7 warnings however I do the see the new list. Here is what i used.
stsadm.exe -o gl-exportlist -url "http://mysite/TSG/Lists/2009 Requisition Spending Log/AllItems.aspx" -filename "c:\spending" -includeusersecurity -versions 4 -nofilecompression -includedescendants
Which works fine Now for import I used

stsadm -o gl-importlist -url "http://mysite/TSG/Lists/2010 Requisition Spending Log/AllItems.aspx" -filename "c:\spending" -includeusersecurity -updateversions 2 -nofilecompression

I just dont see the new list. Help is really appreciated.

MC

Gary Lapointe said...

The URL for the import is just the web URL - you can't use this to change the name of the list unless you've modified the XML in the exported manifest file. Import to a different web and you'll see your list (all you've done is just import on top of your existing list).

Anonymous said...

I want to export a list and import it on a seperated sharepoint server, without access to the source sharepoint server. However the importlist option is giving an error because it can not find it source location.
Is there a solution to realy backup/export and import on a standalone sharepoint server.

Gary Lapointe said...

Try the -excludedependencies parameter.

Andrew Walman said...

Hi Gary,

We have a situation where we need to migrate a subweb to a new site collection, taking the workflow history with it. The standard Stsadm commands are a no-go, would gl-importlist (after we've imported the rest of the site) be a better bet?

Gary Lapointe said...

I'm using the same API that the out of the box export/import commmands use and that API does not support workflow history. You'll need to look to third party commercial tools for that (but be careful as some of them hit the database directly which is will put you in an unsupported state).

OferGal said...

I am trying to export then import on a diferent Farm.
The list is a library of InfoPath files.
Is this ipossible because of the API issue?

Gary Lapointe said...

You should be able to do it but I've not tried that specific scenario (re: infopath)

Anonymous said...

Where can I get the download for gl -exportlist and gl -importlist stsadm commands ?

Gary Lapointe said...

Go to the downloads page and download the appropriate WSP package.

Jonathan said...

Gary, some great tools. Can you tell me what the includeusersecurity parameter does. I thought it would carry over the permissions for each list item, but that doesnt seem to be the case. Is there anyway to carry over the item level permissions? For example I have a list entry that allows read access for a single person, can the gl-export/gl-import replicate the permissions?

Gary Lapointe said...

It's a bug with the content deployment API. Use my export/import list security commands after you import the list.

Jonathan said...

Thank you, works great!

RayD said...

Gary - thanks for an awesome set of tools. I'm battling to use gl-copylist. I'm trying to copy a discussion board called Arb as a test from one site to another in the same site collection, using:
stsadm –o gl-copylist -sourceurl "http://intranet.oldfort.co.za/systems/Lists/Arb/AllItems.aspx" -targeturl "http://intranet.oldfort.co.za/test/"
It appears to complete successfully, but I don't see the Arb discussion board under my Test site. Any ideas? Also tried targeturl of /test/List/, /test/List/Arb etc.

Aaron Copland - Komponist said...

Gary,

great tool set. However, I have the same prob as turtlehand:

The commandline output shows that it exports and imports fine. 0 warnings and 0 errors but when nothing appears in the destination list.

Can you say what's wrong here? I'm really in trouble with my intranet deployment.

Gary Lapointe said...

I'm looking into why the import is reporting success but not actually importing but thus far I can't reproduce so not sure what's going on.

Anonymous said...

Hi Gary,
Is there a way of exporting the lists where the WFE has Forms Based Authentication and in the web.config the "/Lists/" area is set to deny all and allow one user that we can authenticate against? There is anonymous access enabled which is why we have used the deny=* for the "/Lists/" so that if a user browses to the list they are prompted to login and if they are authenticated it will still deny them access.

Gary Lapointe said...

I haven't tried that scenario so I'm not sure if it will work or not. Technically what you're doing is not supported - you should use the lockdown feature and permissions to keep people out of those pages.

Wafi said...

Hi GARY and thank for this great expansion tool,
I have one question plz (& sorry if it posted before):
can I copy list to another existed one?
if yes how, if no how can I import list items to another list?

thanks in advance

Anonymous said...

Hi,

I tried the tool and have a little problem, or I missunderstand this program.

If I use the following command:

stsadm -o gl-copylist -sourceurl "http://www3.sb.local:88/Papers/PaperArchiv" -targeturl "http://www3.sb.local:88/Management/NewPapers/"

The tool always import the List PaperArchiv into the top of Management. But "Newpapers" is a subwebsite and there it should create the copied list.

I tried copylistitem, but then all the original creation filedates are gone. That's not an alternative

Dong said...

Gary,
I am also having a problem copying a library to another site. The structure gets created but the library is empty.

If you need help reproducing the problem, please contact me at ddnguyen@pepco.com. I need to solve this problem.

Thanks
Doug

He Who Has No Clone said...

Hello again, Gary!

I ran into an error when running the export/import commands. It went through the "Progress: Importing User/Group", etc...but when it started the content import, I get this error:

[8/4/2010 10:49:52 AM]: Progress: Importing Group member 1808.
[8/4/2010 10:49:52 AM]: Progress: Starting content import.
[8/4/2010 10:49:52 AM]: Progress: De-Serializing Objects to Database.
[8/4/2010 10:49:52 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions.
[8/4/2010 10:49:53 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/Attachments.
[8/4/2010 10:49:53 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/Discussion.
[8/4/2010 10:49:53 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/Message.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/AllItems.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/Discussion.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/DispForm.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/EditForm.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/Flat.aspx.
[8/4/2010 10:49:54 AM]: Progress: Importing File Lists/taker_discussions/NewForm.aspx.
[8/4/2010 10:49:54 AM]: Progress: Importing File Lists/taker_discussions/Threaded.aspx.
[8/4/2010 10:49:54 AM]: Progress: Importing List tasker_discussions.
[8/4/2010 10:49:54 AM]: Warning: Unable to find a lookup list /111/S3 Tasker_09feb10. The list is not part of the exported package and does not exist in the destination site collection.
[8/4/2010 10:49:56 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/E10-001.
[8/4/2010 10:49:56 AM]: Progress: Importing ListItem /111/staff_sections/S3/Lists/taker_discussions?id=4.
[8/4/2010 10:49:56 AM]: Progress: Importing ListItem /111/staff_sections/S3/Lists/taker_discussions?id=5.
[8/4/2010 10:49:57 AM]: Progress: Import Completed.
[8/4/2010 10:49:57 AM]: Finish Time: 8/4/2010 10:49:57 AM.
[8/4/2010 10:49:57 AM]: Completed with 1 warnings.
[8/4/2010 10:49:57 AM]: Completed with 0 errors.

I'm not sure what went wrong. I exported the document library fine, and then tried to import it to the site that I wanted it to be on. Any ideas?

Gary Lapointe said...

Your library has a lookup column in it which is causing the issues.

Ken said...

Hello again, Gary!

I ran into an error when running the export/import commands. It went through the "Progress: Importing User/Group", etc...but when it started the content import, I get this error:

[8/4/2010 10:49:52 AM]: Progress: Importing Group member 1808.
[8/4/2010 10:49:52 AM]: Progress: Starting content import.
[8/4/2010 10:49:52 AM]: Progress: De-Serializing Objects to Database.
[8/4/2010 10:49:52 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions.
[8/4/2010 10:49:53 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/Attachments.
[8/4/2010 10:49:53 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/Discussion.
[8/4/2010 10:49:53 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/Message.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/AllItems.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/Discussion.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/DispForm.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/EditForm.aspx.
[8/4/2010 10:49:53 AM]: Progress: Importing File Lists/taker_discussions/Flat.aspx.
[8/4/2010 10:49:54 AM]: Progress: Importing File Lists/taker_discussions/NewForm.aspx.
[8/4/2010 10:49:54 AM]: Progress: Importing File Lists/taker_discussions/Threaded.aspx.
[8/4/2010 10:49:54 AM]: Progress: Importing List tasker_discussions.
[8/4/2010 10:49:54 AM]: Warning: Unable to find a lookup list /111/S3 Tasker_09feb10. The list is not part of the exported package and does not exist in the destination site collection.
[8/4/2010 10:49:56 AM]: Progress: Importing Folder /111/staff_sections/S3/Lists/taker_discussions/E10-001.
[8/4/2010 10:49:56 AM]: Progress: Importing ListItem /111/staff_sections/S3/Lists/taker_discussions?id=4.
[8/4/2010 10:49:56 AM]: Progress: Importing ListItem /111/staff_sections/S3/Lists/taker_discussions?id=5.
[8/4/2010 10:49:57 AM]: Progress: Import Completed.
[8/4/2010 10:49:57 AM]: Finish Time: 8/4/2010 10:49:57 AM.
[8/4/2010 10:49:57 AM]: Completed with 1 warnings.
[8/4/2010 10:49:57 AM]: Completed with 0 errors.

I'm not sure what went wrong. I exported the document library fine, and then tried to import it to the site that I wanted it to be on. Any ideas?

Ken said...

I've checked all of the columns that are in there, and I don't see a lookup field anywhere.

For instance, they are all "Date and Time", "Multiple lines of text", "Choice", "Single line of text", or "Person or Group".

Column (click to edit) Type Used in
BDE Suspense Date and Time New Tasker
Comments Multiple lines of text New Tasker
Date Sent Date and Time New Tasker
Event End Date Date and Time New Tasker
Event Start Date Date and Time New Tasker
Fiscal Year Choice New Tasker
Higher Suspense Date and Time New Tasker
MOI Number Single line of text New Tasker
Tasked Organization(s) Choice New Tasker
Tasked Organization(s)_test Person or Group New Tasker
Tasker Title Single line of text Document, New Tasker
Tasking Status Choice New Tasker
Tasking Type Choice New Tasker
Created By Person or Group
Modified By Person or Group
Checked Out To Person or Group

Gary Lapointe said...

Ken - that's odd but the behavior is not uncommon with the deployment API (the thing sucks) - you might try to use the includedescendants and excludedependency parameters to see if maybe you can exclude a dependent item that it's trying to drag along (sounds like that's what's happening).

Ken said...

Thank you very much, Gary!
I'll look into it!

I hope the consulting is working out well for you!

Waiman said...

Hi Gary....saw your tool and love it at first sight....I do need to copy a dozen lists with content across from my test environment to prod. I have a web service which refers to the list in GUID...if I use your tool to copy the lists across will it preserve the GUIDs too? (sorry for being lazy...I know I should try it out first)

Gary Lapointe said...

Waiman - theoretically it should work but for as many scenarios in which it should there are probably at least 10 times more in which it won't - it's one of those things you really do just have to try it out (all comes down to the content deployment API which is riddled with limitations).

Clem said...

Has anyone ever worked through the problem of the "Parent web does not exist"? this occurs with ALL export/import operations regarless of the GL extensions. There is absolutely NO WAY TO EXPORT/IMPORT A LIST OR DOC LIB. While I agree that SharePoint is not a document management system, it should do better than this!

Kathy said...

Question. Is it possible to retain the 'created by' field when doing an export/import of a list?

For example using the following command does not retain the 'created by' field instead the identity is the user that is executing the export/import
stsadm -o gl-exportlistitem -url "https://sitename.domain.com/subsite/blogsite/Lists/Posts/AllPosts.aspx" -filename "d:\posts" -versions 4 -nofilecompression
Thanks Gary for the extensions. Appreciate your hard work in this area.

Gary Lapointe said...

Unfortunately it's a limitation of the content deployment API which I'm just wrapping.

Anonymous said...

Hi Gary,
I have used several of your commands and I am a big fan of them! thanks a lot!
I am trying to use the exportlist and importlist command to move a list out of a site with only the latest version. Then I try to import it back to the same site with only the latest versions. (my main aim here is to delete all the older versions of documents in that list). The export command runs perfectly. However, when I run the import command, it copies all the users, groups etc and then at the stpe to actually import the list, it gives me the error:
"Object reference not set to an instance of an object" as soon as
any suggestions?
I can attach the import log if that might help...

Gary Lapointe said...

I'm not sure why you're getting that error but if all you want to do is delete all the versions then you should be able to just disable versioning for the list (can't remember for sure but I thought that would delete it - if not then a little powershell to loop through all the items and delete the versions should be pretty easy to put together).

Anonymous said...

Thanks for your quick response Gary.
Simply turning off versions prevent new versions from being created int he future, but does not delete existing versions.
I am still a beginner in powershell and havent used it a lot. I was trying to get this done using your commands.
Anyhow, thanks for the reply.

Wayne said...

Hi Gary,

I've just started using your copy list item command to copy the contents from a list on one web app to a list on another.

The export goes through without error, however the import has the following errors...

Progress: Starting content import.
FatalError: Item has already been added. Key in dictionary: '100323 emt minutes.docx' Key being added: '100323 emt minutes.docx'
at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
at System.Collections.Specialized.StringDictionary.Add(String key, String value)
at Lapointe.SharePoint.STSADM.Commands.Lists.ImportListItem.OnImportStarted(Object sender, SPDeploymentEventArgs e)
at Microsoft.SharePoint.Deployment.SPDeployment.OnStarted(SPDeploymentEventArgs e)
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 10/25/2010 4:37:03 PM.
Completed with 0 warnings.
Completed with 1 errors.

The command I am running is:

stsadm -o gl-copylistitem -sourceurl http:///Forms/AllItems.aspx -targeturl http:///Forms/AllIt
ems.aspx -versions 4 -updateversions 2 -includeusersecurity -suppressafterevents

You will notice I have not used the switch to halt on fatal error, yet it does indeed halt.

Any suggestions on where to look. Would be happy if the copy just skipped this file.

Thanks,
Wayne.

Jasper said...

Hi Gary, thanks for your great tools.

I try to Copy with
stsadm –o gl-copylist -sourceurl "http://intranet/Documents/Forms/AllItems.aspx" -targeturl "http://teamsites/" -includeusersecurity -updateversions 2 -versions 4 -retargetlinks

but I get an error
"Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt" (German)
"The Object Reference wasn't set to an Object instance" (So far in english)

Do you know a solution or what I do wrong?

Thanks again.

Jasper

Swarup said...

Gary, these have been really useful!

We were running WSS3.0 and using your copy list function to archive some large lists we had.

We moved to Sharepoint Foundation and I am now getting this error

"The Web application at http://portal/orders/Test/Forms/AllOrders.aspx could not be found. Verify that you have typed the URLK correctly. If the URL should be serving existing content, the system administrator may need to add a new request URL mapping to the intended application."

I've been running this from the server itself, running as administrator. the server is a w2k8 r2 box (64 bit).

what could i be doing wrong? permissions?

Swarup said...

I removed the old solution and installed the Sharepoint foundatino 2010 solution. I'm now getting the same error that Jasper was/is getting

(The Object Reference wasn't set to an Object instance)