Difference between revisions of "Xyphoid/MediawikiConversion"

From Exalted - Unofficial Wiki
Jump to: navigation, search
m (link fix)
m (Script: fix links messed up in conversion)
 
(50 intermediate revisions by 12 users not shown)
Line 1: Line 1:
So, hosting usemod is becoming harder and harder. I'm trying out the mediawiki site conversion tools, as it would be much easier to host that way. Comments? Feedback? - [[Xyphoid]]
+
(pre-conversion discussion at [[Xyphoid/MediawikiConversionHistory]])
  
=== Comments ===
+
== Post-conversion ==
What makes it harder? I use both this wiki and [[MediaWiki]] pretty extensively, and I must say that this place seems snappier and cleaner, at least. Plus, there's the advantage of not having to re-write a bunch of the sections to fit the different syntax. At the same time, it's your server and time and money. Is it security updates, or some such? I would've guessed that this place didn't need much work anymore... -- [[GreenLantern]]
 
  
The server it's currently running on will be retired in the near future, so I have to do some work in either case. I have a couple of choices - one  is find another place to run usemod, the other is migrate to something else. Usemod is a perl CGI script writing to disk - this is pretty much unique among the stuff we have, and it doesn't suit our load-balancing and configuration. Mediawiki is php+mysql, which is what our main cluster is designed for. Basically I can keep mediawiki with the rest of my hosted content, but soon I won't have a good place for usemod.
+
Just fyi (again) the links under nikink/characters are broken since the media wiki thinks they should translate to nikink/characters/charactername but the pages are actually on nikink/charactername. Minor, but could catch the unwary. [[nikink]] - also, spam! Already!?!? Making logins active will have to be a priority I guess. :(
 +
: Can you create an account? I'm going to add the recaptcha spam filter first up [[User:Xyphoid|Xyphoid]] 03:50, 6 April 2010 (UTC)
 +
:: This has been added - you now require a captcha to add an extenral link, or register an account. [[User:Xyphoid|Xyphoid]]
 +
: Nope - php errors when attempting to create an account. And darn it, so many of my pages are borked! Maybe I was creating them wrong in the first place? Hm. Ah well. [[nikink]] <i>Also, running slow. Is that something to get used to with this version of software+hardware? Just curious. </i>
 +
: Yep, account is now created. Adding the captcha fixed it. :) [[nikink]]
 +
:: php errors were probably mid-captcha-installation. Speed, we'll see how it goes - there's more going on now, and this is on shared hosting now instead of having an entire server to itself. [[User:Xyphoid|Xyphoid]]
  
Another option is that if someone else has a good place to host usemod wikis, I can pass it over to them.  
+
I will have a look at bulk conversion of the /characters/charactername issue, whoops. [[User:Xyphoid|Xyphoid]]
 +
: Sweet, it'd be great if that could be automated - my campaign pages have the same issue, and there's a LOT of links from them! Thanks for looking into it! :) [[nikink]] PS: and should we move our previous userpages to our new User:Name pages? Or just leave them as relics? Can they be redirected?
 +
:: There may be a delay until the 22nd or so with this since I'm on holiday shortly, but I'll have a shot. You can redirect a page by putting <nowiki>#REDIRECT [[somelink]]</nowiki> as the only content - so you could redirect in either direction as you prefer. As there's no way of telling what's a userpage unless people actually create accounts, and there are so many historical userpages, I won't be automatically moving them. The main advantage to moving to the User:xx version is that you can take advantage of the autolinks from recent changes, and from typing 3 tildes in an edit. [[User:Xyphoid|Xyphoid]]
 +
::I will have to keep playing with it. All very new and unfamiliar. :) I've notice tables are all broken, I suppose that's a minor detail; just need to learn to table in mediawiki. The list of orphaned pages is quite helpful in the meantime while you work on the scripted fix for the incorrect links. :) [[nikink]]
  
I ran a test of the conversion and it appears promising at first glance - http://exalted-new.xi.co.nz/exwiki/index.php/WelcomeToExaltedWiki - note that any changes made to that copy will be lost for now as there were some errors reading pages starting with M. There are some obvious layout errors to work out, I'd be interested to hear about any conversion issues with particular pages that people spot (broken links, broken wiki formatting conversion) -- [[Xyphoid]]
+
Whoa!  Scared the daylights out of me when I went to edit my userpage, open from a while ago, and it was no longer there!  Stuff looks absolutely great; I noticed that some of the charm formatting gets broken by the changeover.  Is there a new text formatting or help page set up for what we should or should not do with the new system as opposed to the old?  Man, and I was terrified enough trying to think of a way to edit and duplicate my stuff for the WW wiki; oh well, gives me a chance to go through and double check mechanics -- [[IsawaBrian]]
 +
:There is an editing help button at the bottom of the mediawiki edit page. As for how to use it I think that's going to be an evolutionary process - I am way out of date as to how y'all are using this wiki, I've been mostly playing D&D4 for the last few years... [[Xyphoid]]
 +
::Sadly, Help:Editting is a no-text page at the moment.  Don't worry about it for now, though, man.  Enjoy your wedding and the time after.  Congratulations and felicitations; we'll be here when things achieve your new normal. I'll just see if I can find a commonality in the breakages. -- [[IsawaBrian]]
 +
:::I've copied the editing page from the main mediawiki page over - some broken links but that's fixable. [[User:Xyphoid|Xyphoid]]
  
I've just checked out my userpage as a first step, and first thing to notice is that any link on my userpage that starts with / is broken. It seems the pages actually exist if I type into the url bar, "http://exalted-new.xi.co.nz/exwiki/index.php/Nikink/TrialByFireNotes" but the mediawiki obviously doesn't recognise the '/' format. I'm surprised the conversion worked so well actually, well done! :) [[nikink]]
+
So apologies for the quick changeover - I'm going to be at my wedding from Friday and away for a few weeks, and the server was going to probably be down during some of that time. I'll be seeing what I can get fixed before I leave, and then will catch up when I'm back. [[Xyphoid]]
:I also observed this phenomenon. A simple solution is to enter the name of the page you are slash-linking off of in front of the dash, e.g. <nowiki>[[Nikink/TrialByFireNotes]]</nowiki> as opposed to just <nowiki>/TrialByFireNotes</nowiki>. So it's just another layout problem that might need to be fixed by hand. - [[Han'ya]]
+
 
::I am going to have a shot at modifying the export/import tool to fix these up - it's not really practical to fix all of these by hand. [[Xyphoid]]
+
Congrats on your wedding!  When you get back, it would be helpful if you could make it so subpages contain backlinks to the parent page in the Main namespace, like they do in the user Namespace - [[User:Kraken|Kraken]]
 +
:The issue is that subpages don't actually exist in mediawiki - they're just pages with slashes in their name. I'll have a poke at writing an extension to automatically create parent links. Should it go all the way to the top? (eg should the page <nowiki>[[Red/Green]]</nowiki> link back to <nowiki>[[Red]]</nowiki> or <nowiki>[[Red/Blue]]</nowiki>? [[User:Xyphoid|Xyphoid]]
 +
:: Could you do both? <nowiki>[[Red/Green]]</nowiki> links back to <nowiki>[[Red]]</nowiki> and <nowiki>[[Red/Blue]]</nowiki>, while <nowiki>[[Red/Blue]]</nowiki> links back to <nowiki>[[Red]]</nowiki>? I guess that depends on how many /'s can exist in one path. I haven't yet seen more than two, but I don't know definitively. [[nikink]]
 +
::: The reason I brought it up is [[User:Kraken/BP|this page]].  For User subpages, it seems to do so already.  So something which did the breadcrumbs style thing for any page would be great.  And the code should be there.  Somewhere.  [[User:Kraken|Kraken]]
 +
 
 +
*I don't know where to put this, so I will place this here. All the links I posted on my userpage are red links in the new document. Is everything I had lost? Or are they just moved till they can be found. Because seriously, I very much do not want to have to go over it all again just because there was a need to put the completely usable wiki into a new format. This is my user page, http://exalted.xi.co.nz/wiki/Heru . Assistance would be apprecai8ted.
 +
:No need to panic, you haven't lost anything. In the section immediately below this you can see an area for links that need fixing. The links on your pages probably look like those. If they don't, add them to the list so Xyphoid can work on a script for them. In the meantime, you can get to your material by removing the middle section of the path. So <nowiki>[[Nikink/MettanAku]]</nowiki> is wrong link, but <nowiki>[[Nikink/MettanAku]]</nowiki> is correct address. Also, this wasn't a transition made 'on a whim', or just because Xyphoid felt mean. He had valid reason for the switch and the suddenness of it. :) [[nikink]]
 +
::Yep, my apologies for the relatively abrupt transition, but the alternative was a high chance of the wiki being down completely for several weeks while I was away. [[Xyphoid]]
 +
::: Also, I know a few people were after a dump of the entire wiki for backup / peace of mind purposes - there is a file 'exwiki_import.xml' in the root of the site, which can be imported directly into mediawiki. Let me know if you need a more up to date version. [[Xyphoid]]
 +
 
 +
A few comments on this conversion:
 +
 
 +
# This ship has probably already sailed, but it would have been easier to just enable subpages in the main namespace (<code>$wgNamespacesWithSubpages[NS_MAIN] = true;</code>) than to try to translate them all.
 +
# There are a lot of links ''into'' this wiki on outside sites like forums and such. Following them no longer leads to the correct place; however, it would be pretty easy to add a <code>.htaccess</code> rule to take care of this. Something like <code>RedirectMatch 301 wiki.pl?(.*)$ http://exalted.xi.co.nz/wiki/$1</code> would probably work.
 +
# You might want to fill in the [[MediaWiki:Common.css]] page with the contents of [http://www.mediawiki.org/wiki/MediaWiki:Common.css this page]. It provides some common table styles and such that are fairly useful. ('''Update''' Looks like I have permission to do this, somehow, so I have done it.)
 +
# I will change the [[InterPageLinks]] page, as this feature should now work, using slightly different markup.
 +
# It might be useful to start making use of the "Discussion" pages that MediaWiki supports. I guess the notion would be to slowly migrate the "comments" section that most pages have into the discussion page. Or, we could just ignore it (in which case, I think you can use a preference to turn the feature off).
 +
# I spent some time a few years ago thinking about this transition. One problem is user pages. By default, MediaWiki creates a page like [[User:Wordman]] for each new user; however, this plays merry hell with the years of history in the old wiki. I looked to see if it is possible to just turn the User:* space off. I think it is possible, but I never looked that far into it. In any case, I suggest we totally ignore the User:* pages and keep the user pages that already existed. (I set my [[User:Wordman]] to redirect to [[Wordman]]. I suggest others do something similar.)
 +
# At some point, a discussion about if/how to use Catagories is probably in order.
 +
# Some extensions could be helpful, but this is a tough call. For example, I find the [http://semantic-mediawiki.org/wiki/Semantic_MediaWiki Semantic plugins] supremely useful for my [http://rpg.divnull.com/wiki/index.php/Special:Version own wiki], but using them takes discipline, and it is ponderous to retrofit old content to take advantage of them, so may not be appropriate here.
 +
# I think it is possible to enable WikiWordSyntax in MediaWiki. Given the long history of this site, this could be desirable.
 +
# The [[MediaWiki:Sidebar]] should be changed to have, at least, the [[WikiContent]] link. ('''Update''' Looks like I have permission to do this, somehow, so I have done it.)
 +
 
 +
- [[Wordman]]
 +
 
 +
Some of my pages don't seem to have linked up.  How do I find them? -- [[BrilliantRain]]
 +
: Check the 'Links that need fixing' section below. If your links look like any of those examples you just need to follow the example to fix them. If your links do not look like any of those examples, you should probably add a new example to the list so that eventually (not too far away) Xyphoid can run a conversion script to fix all examples. Make sense? [[nikink]] <i>PS: I have gone to [[BrilliantRain/Artifacts]] and left an example for you.</i>
 +
::That did it, thanks! -- [[BrilliantRain]]
 +
 
 +
Started the [[Wordman/ProjectGezlak‎|Project Gezlak‎]] to fix these links automatically, since this is the two month mark of the conversion, and no other action has been taken to fix them. - [[Wordman]]
 +
 
 +
== Links needing fixing ==
 +
 
 +
Please list types of links that need to be fixed by scripting, along with an example page.
 +
 
 +
* <nowiki>[[FrivYeti/SaGHistory1|The First Age; From the Fall of the Primordials to the Great Contagion.]]</nowiki> - non-subpage links with descriptions, see [[FrivYeti/SteelAndGlass]]. Looks like the import script wasn't prepared for links with spaces around the pipe. That is, stuff like <nowiki>[[SamplePage|a sample page]]</nowiki> were handled correctly, but <nowiki>[[SamplePage|a sample page]]</nowiki> got totally mangled.
 +
 
 +
* <nowiki>[[Nikink/MettanAku]]</nowiki> - <nowiki>[[/MettanAku]]</nowiki> links on [[Nikink/Characters]], it appears /xxx should always link to the top level page even on zzz/yyy. That is, on page <nowiki>Nikink/Characters</nowiki>, a link like <nowiki>/MettanAku</nowiki> should have become <nowiki>[[Nikink/MettanAku]]</nowiki>. This screwed up ''a lot'' of links.
 +
 
 +
* <nowiki>Heru/[[SubpageName]]</nowiki> - <nowiki>Heru/SubpageName</nowiki> I think the presence of <nowiki>[[...]]</nowiki> in Heru's subpage links confused the conversion script.
 +
 
 +
* Artifacts page has broken links to most subpages. Things presented in the format <nowiki>[[Artifacts/DefensiveLevel1|Level 1 Artifacts]]</nowiki> are broken. Not sure if it's the wrong number of brackets or if it's just a format that worked on the old wiki and doesn't on this one.
 +
 
 +
* Pages that originally used HTML tags in ''capital'' letters (e.g. <tt><nowiki><B>bold!</B></nowiki></tt>) got fairly mangled by the original conversion script. For example, a page like [[DragonBloodedMelee/TheMyriadOfShades‎‎]] had some text like <tt><nowiki><B>Cost:</B></nowiki></tt> that got turned into <tt><nowiki><B>Cost:</B></nowiki></tt>
 +
 
 +
== Other requests ==
 +
 
 +
* Automatic backlinks from child pages (anything of the name xxx/xxx) to the parent page

Latest revision as of 05:02, 9 June 2010

(pre-conversion discussion at Xyphoid/MediawikiConversionHistory)

Post-conversion

Just fyi (again) the links under nikink/characters are broken since the media wiki thinks they should translate to nikink/characters/charactername but the pages are actually on nikink/charactername. Minor, but could catch the unwary. nikink - also, spam! Already!?!? Making logins active will have to be a priority I guess. :(

Can you create an account? I'm going to add the recaptcha spam filter first up Xyphoid 03:50, 6 April 2010 (UTC)
This has been added - you now require a captcha to add an extenral link, or register an account. Xyphoid
Nope - php errors when attempting to create an account. And darn it, so many of my pages are borked! Maybe I was creating them wrong in the first place? Hm. Ah well. nikink Also, running slow. Is that something to get used to with this version of software+hardware? Just curious.
Yep, account is now created. Adding the captcha fixed it. :) nikink
php errors were probably mid-captcha-installation. Speed, we'll see how it goes - there's more going on now, and this is on shared hosting now instead of having an entire server to itself. Xyphoid

I will have a look at bulk conversion of the /characters/charactername issue, whoops. Xyphoid

Sweet, it'd be great if that could be automated - my campaign pages have the same issue, and there's a LOT of links from them! Thanks for looking into it! :) nikink PS: and should we move our previous userpages to our new User:Name pages? Or just leave them as relics? Can they be redirected?
There may be a delay until the 22nd or so with this since I'm on holiday shortly, but I'll have a shot. You can redirect a page by putting #REDIRECT [[somelink]] as the only content - so you could redirect in either direction as you prefer. As there's no way of telling what's a userpage unless people actually create accounts, and there are so many historical userpages, I won't be automatically moving them. The main advantage to moving to the User:xx version is that you can take advantage of the autolinks from recent changes, and from typing 3 tildes in an edit. Xyphoid
I will have to keep playing with it. All very new and unfamiliar. :) I've notice tables are all broken, I suppose that's a minor detail; just need to learn to table in mediawiki. The list of orphaned pages is quite helpful in the meantime while you work on the scripted fix for the incorrect links. :) nikink

Whoa! Scared the daylights out of me when I went to edit my userpage, open from a while ago, and it was no longer there! Stuff looks absolutely great; I noticed that some of the charm formatting gets broken by the changeover. Is there a new text formatting or help page set up for what we should or should not do with the new system as opposed to the old? Man, and I was terrified enough trying to think of a way to edit and duplicate my stuff for the WW wiki; oh well, gives me a chance to go through and double check mechanics -- IsawaBrian

There is an editing help button at the bottom of the mediawiki edit page. As for how to use it I think that's going to be an evolutionary process - I am way out of date as to how y'all are using this wiki, I've been mostly playing D&D4 for the last few years... Xyphoid
Sadly, Help:Editting is a no-text page at the moment. Don't worry about it for now, though, man. Enjoy your wedding and the time after. Congratulations and felicitations; we'll be here when things achieve your new normal. I'll just see if I can find a commonality in the breakages. -- IsawaBrian
I've copied the editing page from the main mediawiki page over - some broken links but that's fixable. Xyphoid

So apologies for the quick changeover - I'm going to be at my wedding from Friday and away for a few weeks, and the server was going to probably be down during some of that time. I'll be seeing what I can get fixed before I leave, and then will catch up when I'm back. Xyphoid

Congrats on your wedding! When you get back, it would be helpful if you could make it so subpages contain backlinks to the parent page in the Main namespace, like they do in the user Namespace - Kraken

The issue is that subpages don't actually exist in mediawiki - they're just pages with slashes in their name. I'll have a poke at writing an extension to automatically create parent links. Should it go all the way to the top? (eg should the page [[Red/Green]] link back to [[Red]] or [[Red/Blue]]? Xyphoid
Could you do both? [[Red/Green]] links back to [[Red]] and [[Red/Blue]], while [[Red/Blue]] links back to [[Red]]? I guess that depends on how many /'s can exist in one path. I haven't yet seen more than two, but I don't know definitively. nikink
The reason I brought it up is this page. For User subpages, it seems to do so already. So something which did the breadcrumbs style thing for any page would be great. And the code should be there. Somewhere. Kraken
  • I don't know where to put this, so I will place this here. All the links I posted on my userpage are red links in the new document. Is everything I had lost? Or are they just moved till they can be found. Because seriously, I very much do not want to have to go over it all again just because there was a need to put the completely usable wiki into a new format. This is my user page, http://exalted.xi.co.nz/wiki/Heru . Assistance would be apprecai8ted.
No need to panic, you haven't lost anything. In the section immediately below this you can see an area for links that need fixing. The links on your pages probably look like those. If they don't, add them to the list so Xyphoid can work on a script for them. In the meantime, you can get to your material by removing the middle section of the path. So [[Nikink/MettanAku]] is wrong link, but [[Nikink/MettanAku]] is correct address. Also, this wasn't a transition made 'on a whim', or just because Xyphoid felt mean. He had valid reason for the switch and the suddenness of it. :) nikink
Yep, my apologies for the relatively abrupt transition, but the alternative was a high chance of the wiki being down completely for several weeks while I was away. Xyphoid
Also, I know a few people were after a dump of the entire wiki for backup / peace of mind purposes - there is a file 'exwiki_import.xml' in the root of the site, which can be imported directly into mediawiki. Let me know if you need a more up to date version. Xyphoid

A few comments on this conversion:

  1. This ship has probably already sailed, but it would have been easier to just enable subpages in the main namespace ($wgNamespacesWithSubpages[NS_MAIN] = true;) than to try to translate them all.
  2. There are a lot of links into this wiki on outside sites like forums and such. Following them no longer leads to the correct place; however, it would be pretty easy to add a .htaccess rule to take care of this. Something like RedirectMatch 301 wiki.pl?(.*)$ http://exalted.xi.co.nz/wiki/$1 would probably work.
  3. You might want to fill in the MediaWiki:Common.css page with the contents of this page. It provides some common table styles and such that are fairly useful. (Update Looks like I have permission to do this, somehow, so I have done it.)
  4. I will change the InterPageLinks page, as this feature should now work, using slightly different markup.
  5. It might be useful to start making use of the "Discussion" pages that MediaWiki supports. I guess the notion would be to slowly migrate the "comments" section that most pages have into the discussion page. Or, we could just ignore it (in which case, I think you can use a preference to turn the feature off).
  6. I spent some time a few years ago thinking about this transition. One problem is user pages. By default, MediaWiki creates a page like User:Wordman for each new user; however, this plays merry hell with the years of history in the old wiki. I looked to see if it is possible to just turn the User:* space off. I think it is possible, but I never looked that far into it. In any case, I suggest we totally ignore the User:* pages and keep the user pages that already existed. (I set my User:Wordman to redirect to Wordman. I suggest others do something similar.)
  7. At some point, a discussion about if/how to use Catagories is probably in order.
  8. Some extensions could be helpful, but this is a tough call. For example, I find the Semantic plugins supremely useful for my own wiki, but using them takes discipline, and it is ponderous to retrofit old content to take advantage of them, so may not be appropriate here.
  9. I think it is possible to enable WikiWordSyntax in MediaWiki. Given the long history of this site, this could be desirable.
  10. The MediaWiki:Sidebar should be changed to have, at least, the WikiContent link. (Update Looks like I have permission to do this, somehow, so I have done it.)

- Wordman

Some of my pages don't seem to have linked up. How do I find them? -- BrilliantRain

Check the 'Links that need fixing' section below. If your links look like any of those examples you just need to follow the example to fix them. If your links do not look like any of those examples, you should probably add a new example to the list so that eventually (not too far away) Xyphoid can run a conversion script to fix all examples. Make sense? nikink PS: I have gone to BrilliantRain/Artifacts and left an example for you.
That did it, thanks! -- BrilliantRain

Started the Project Gezlak‎ to fix these links automatically, since this is the two month mark of the conversion, and no other action has been taken to fix them. - Wordman

Links needing fixing

Please list types of links that need to be fixed by scripting, along with an example page.

  • [[FrivYeti/SaGHistory1|The First Age; From the Fall of the Primordials to the Great Contagion.]] - non-subpage links with descriptions, see FrivYeti/SteelAndGlass. Looks like the import script wasn't prepared for links with spaces around the pipe. That is, stuff like [[SamplePage|a sample page]] were handled correctly, but [[SamplePage|a sample page]] got totally mangled.
  • [[Nikink/MettanAku]] - [[/MettanAku]] links on Nikink/Characters, it appears /xxx should always link to the top level page even on zzz/yyy. That is, on page Nikink/Characters, a link like /MettanAku should have become [[Nikink/MettanAku]]. This screwed up a lot of links.
  • Heru/[[SubpageName]] - Heru/SubpageName I think the presence of [[...]] in Heru's subpage links confused the conversion script.
  • Artifacts page has broken links to most subpages. Things presented in the format [[Artifacts/DefensiveLevel1|Level 1 Artifacts]] are broken. Not sure if it's the wrong number of brackets or if it's just a format that worked on the old wiki and doesn't on this one.
  • Pages that originally used HTML tags in capital letters (e.g. <B>bold!</B>) got fairly mangled by the original conversion script. For example, a page like DragonBloodedMelee/TheMyriadOfShades‎‎ had some text like <B>Cost:</B> that got turned into <B>Cost:</B>

Other requests

  • Automatic backlinks from child pages (anything of the name xxx/xxx) to the parent page