LyZ: LyX plugin for Zotero

LyZ is a plugin for Zotero, which is intended to make working with LyX/Zotero more pleasant.

Features

  • Inserting citations to LyX from Zotero.
  • BibTeX database automatically updated when citation is inserted.
  • BibTeX database can be updated when the references in Zotero are modified.Get Zotero
  • Custom BibTeX key format.
  • Unique BibTeX key check.
  • Custom BibTeX translators.
  • Synchronization of BibTex keys between LyX document, BibTeX database and LyZ.
  • Editing of master/child documents (the same BibTeX database) and multiple documents (different BibTeX database for each document).
  • Persistent association of LyX documents and BibTeX databases.
  • Support for group cooperation.

Installation and Settings

  1. LyZ works with Zotero 2.0 and later.
  2. Install LyZ  from Firefox Addons and restart Firefox.
  3. Open LyX and set LyXServer path (in LyX go to menu, Tools > Preferences... > Paths and set LyXServer pipe to): Windows users can use the default path setting in LyZ, i.e. “\\.\pipe\lyxpipe”. Linux and Mac users can use e.g. “~/.lyxpipe” and change the LyZ settings accordingly.
  4. By default all characters are escaped, e.g. š becomes \v{s}. Set extensions.lyz.use_utf8 to true in about:config to avoid escaping (in case you need to use non-latin scripts such as Chinese).
  5. Open document in LyX.

Adding and inserting citations

  1. Open Zotero, select citation and either from LyZ menu or from context menu (right click on the citation) do “Cite in LyX”.
  2. When you try to insert citation into a new document (i.e. document unknown to LyZ) you have to select BibTeX database. You will be asked to either create a new database or choose a database you have created before (by LyZ). Type the database name (*.bib extension is automatically added if you don’t provide it) and press OK. The citation is added into the BibTeX database you have just created and inserted into the LyX document. Your citation should appear in LyX document now.
  3. Insert the BibTeX database into LyX document. See BibTeX and LyX.
  4. Insert additional citations, new ones from Zotero, previously added citation can be either inserted through the LyX interface or from Zotero.
  5. If you modify the reference information in Zotero, use “Update BibTeX” from LyZ menu to update the BibTeX database.

Custom BibTeX keys

  1. Custom BibTeX keys can be set in LyZ > Settings dialog.
  2. The default format is “author year title”. Good alternative for the use with natbib package is format “zotero”, which will produce “human unreadable” BibTeX keys. It consists of library ID and zotero item key, e.g. 0_XXXXX. Format keyword “zoteroShort” will produce only XXXXX, which can be used when exporting from OOo.
  3. Key formats can be composed from keywords “author”, “year”, “title”, “zotero” and separators such as “_” (the following characters should be safe to use: a-z, 0-9!$&*+-.:;<>?[]^`|), e.g. “author year title”, which will produce authoryeartitle BibTeX key, e.g. tichy1988thefoundations. If you prefer the default format used in Zotero BibTeX export (tichy_thefoundations_1988), change the pattern to “author _ title _ year”. The keywords and any separating characters have to be separated by space. Characters in citekeys are escaped and unsafe characters are removed conforming to /[^a-z0-9\!\$\&\*\+\-\.\/\:\;\<\>\?\[\]\^\_\`\|]+/g.
  4. If you use custom BibTeX translator and want to use the cite keys generated in the translator, go to LyZ > Setting and switch creation of cite keys off.

Update BibTeX

Menu command “Update BibTex” will update BibTeX database of the active LyX document. Whatever changes made in Zotero will be reflected in the update, including BibTeX keys if the BibTeX key format or the relevant information (first author, title, year) have been changed.

When a single BibTeX database is shared among several authors, e.g. using version control system such as SVN and CVS, LyZ database is updated from local working copy of BibTeX database.

Only current LyX document will be updated. Multiple document update is implemented, but seems unreliable. Needs more testing…

Synchronization between LyX/BibTeX/LyZ

When Zotero reference is changed, BibTeX database as well as the LyX document can be updated. This is useful when nice BibTeX keys are used, such as ‘author year title’ and e.g. the title or the first author’s surname has been modified, which will result in modification of the BibTeX key. When the “Update BibTeX” is run, you should select to update the LyX document in case your modifications to Zotero references might affect your BibTeX keys. Before the update, a backup of the active LyX document is made (extension *.lyz). LyZ will save, close and then reopen your document.

Notes on usage

  1. Credits: Lyz is inspired by Lytero, which I wanted to improve, but ended up starting from scratch. Hence the new name. Thanks to Dan Stillman and mronkko at zotero.org forum. Justin Wood for BOM and escaping suggestions and donation!
  2. Always modify your references in Zotero. If you modify the BibTeX database manually or using other software, LyZ will overwrite all the changes when you run “Update BibTeX“.
  3. Keep your Zotero items unique. LyZ does not make an attempt to identify duplicates, they will be treated as different items and unique BibTeX keys will be created.
  4. When working with master/child documents in LyX, you will have to specify the BibTeX database for each child document. This will be the same file as specified for the master document. Unfortunately LyZ has no way of knowing the relationship between documents, so it has to be done manually.
  5. Old writing projects can be deleted from LyZ menu. When the record of BibTeX database is deleted, all associated LyX document records are also deleted. When LyX document is deleted, associated BibTeX database will remain untouched as it may be used by another document. The actual file won’t be deleted.
  6. BibTeX database is uniformly exported to UTF8 and all characters are escaped. Exporting to doesn’t seem to have any advantages. When your references contain extra characters such as the degree symbol, add \usepackage{textcomp} to your preamble.
  7. URLs in bibliography. Use bibliography style that can handle URLs, such as apalike, or add \usepackage{url} to your preamble.
  8. LyZ stores the information about documents and BibTeX databases in lyz.sqlite in the Zotero folder. If you backup Zotero with some synchronization application, you will notice the new file.
  9. When working with non-latin scripts such as Chinese, set extensions.lyz.use_utf8 to true in about:config to avoid escaping.

Source code

Please explore at https://github.com/willsALMANJ/lyz.

Problems, Ideas, Requests

Please follow these tips to find out the source of your problem:

  1. Run Lyx in debugging mode from commandline by (Linux and Mac users change accordingly)

    lyx.exe -dbg lyxserver

    Useful messages should appear whenever you send some command from Lyz. A useful message will contain words like ERROR, undefined etc.

  2. To send a command from Lyz, you simply use one of the functions, e.g. “Cite in Lyx” or use command line, viz. http://wiki.lyx.org/LyX/LyXServer.
  3. Also you can use a variant of the last version of Lyz, which has additional menu item “Test…”. This will allow you to send single commands to Lyx. Currently Lyz uses the following commands (code in the parentheses indicates usage):
    • server-get-filename (server-get-filename)
    • citation-insert (citation-insert:LYZ:author2010title)
    • buffer-next (buffer-next)
    • server-get-xy (server-get-xy)
    • buffer-write (buffer-write)
    • buffer-close (buffer-close)
    • file-open (file-open:path, where path is the response from server-get-filename)

    All functions Lyz has are patched together using these commands. The most trouble some seems to be the server-get-filename.

  4. Extract useful information from Firefox’s Error Console and from commandline where you ran lyx.exe -dbg lyxserver and send it to me or even better file a bug, ask a question or submit a request at https://github.com/willsALMANJ/lyz

 

When Zotero reference is changed, BibTeX database as well as the LyX document can be updated. This is useful when nice BibTeX keys is used, such as ‘author year title’ and e.g. the title or the first author’s surname has been modified.

192 comments to LyZ: LyX plugin for Zotero

  • Marsh

    Tom, Petr, and all the ships at sea,

    I had the same problem as Tom but have the very excellent MR Tech Toolkit plug-in installed in Firefox. It allows you to right-click an entry in the FF add-ons list and choose a context menu item to make the installed add-on compatible with the current version of FF. Of course, the add-on may really be incompatible and work in ways that break things, but usually the incompatibility is just because of the way maxVersion is set. This solution seemed to work fine.

    Then today I upgraded LyZ to version 1.6.7.1, and there’s no problem any longer (w/ FF 3.6.3).

    So that’s the solution. Now for the problem.

    For some reason, FF did not update LyZ to 1.6.7.1 automatically. I even went into Add-ons, highlighted LyZ, and pressed Update, but still I got zilch. I had to go to the web site and install the new version manually and without benefit of an update.

    Any chance of getting this last problem fixed?

    Marsh

  • Petr

    Marsch and Tom,
    unfortunately until Lyz is moved from experimental, the automatic updates won’t work. There are suggestions to use https://addons.mozilla.org/en-US/firefox/addon/59266, which is also experimental.
    I have applied to move Lyz to public some time ago, but the review process takes time.
    We must be patient :)
    Petr

  • john

    Pretty good software, exactly what I’ve been searching for. Many thanks!
    One feature request: At least for me it seems not to be possible to remove an entry once you decided to push it from zotero to Lyx.
    Removing the entry out of the bibtext file will restore the entry via the next “update bibtex” in zotero, removing the item in zotero completely (delete the entry and empty the trash) seems to crash Lyz somehow, as now Lyz won’t update the bibtex file any longer and also refuses to update the Lyx file. Removing the entry out of the bibtex file manually after deleting it from zotero won’t make Lyz work again.
    So an entry once cited in Lyx needs to stay in zotero forever and also can’t be deleted in the bibtex file, which may get us in messy trouble somehow, especially as purging an item in zotero crashes Lyz (tried it 2 times).
    Regards and thx again
    John

  • john

    update: what is still working after deleting an entry in zotero is pushing new entrys to Lyx out of zotero. However updating the bibtex file won’t work, neither old entries before the deletion are updated, nor are the most recent ones. The bibtex file won’t be changed any longer via “update bibtex”

    • Petr

      John, good point. The ‘update bibtex’ works via the keys (which are zotero unique ids) that you find on the first line of the generated bibtex file. So lyz basically just reads those and everything else is rewritten.
      I will add checking for deleted items soon. Stay tuned.
      Thanks a lot

  • john

    Dear Petr,

    many thanks for your quick answer. Unfortunately, I got another feature request, perhaps very special, but perhaps also

    really useful for many people:

    I am using Lyx and Zotero at least on 3 Computers (2xWin, 1 Linux) and I am syncing the zotero folder and the Lyx and corresponding bibtex file I am working on to all these machines (via dropbox; btw: great piece of software, too).

    So I have to tell Lyz on the first time on every computer which Lyx and which bib file to use.
    So for now, I got 3 entries, all pointing to another path but in effect to the same files (lyx and bibtex). At first, it seemed to work. I can cite in Lyx from Zotero on all machines.
    But if I update the bibtex file via Lyz, all inserted entrys from other computers into Lyx become [undefined] even though in
    effect the bibtex file is still the same and even though the identifiers in Lyx haven’t changed.

    To cut a long story short, it would be perhaps easier to tell Lyz to use not an absolute path to the Lyx and bibtex file
    (like: C:\documents and settings….), but to use a relative one, like: example.lyx needs the bibtex file example.bib in the
    same folder, or the folder “references” as a subfolder of the folder the lyx file is in. Then one would only have one entry
    for a bibtex or Lyx file and only one sqlite entry.

    Lyx does this too with images inserted into lyx if the images are in the same or in a subfolder of the lyx file. Using my
    suggested solution, one would only have one entry in the sqlite database and so Lyz should work on all Computers even if you
    carry your files around what many people might actually do.

    Another problem:

    Lyz seems to loose connection to Lyxpipe somehow: Using Ubuntu Lucid and Lyx 1.6.5 I can usually add only one entry, after that, firefox just hangs. I even can’t update the bibtex file. Only closing Lyx will get firefox working again. Then the error message is

    ERROR: lyxGetDoc:
    Could not retrieve document name.

    After adding some entries in Windows 7, trying to update the bibtex file led to nearly the same error, with the difference
    that the first error message was something about no connection to the lyxpipe. However, firefox does not crash, you can make
    another try right after clicking away the error messages. In Ubuntu however, you only can close Lyx to unfreeze firefox
    again. Only one time firefox told me sth. like “a script is working but needs to long”. Unfortunately there aren’t any error
    messages in firefox’ Error console.
    It does not seem to have sth. to do with using Lyz, Zotero, and my files via Dropbox on different computers as I already
    tried all this stuff with fresh mozilla user profiles, fresh lyx configurations, fresh Zotero etc.

    Update: After trying again with a fresh profile on Ubuntu with only 3 Zotero entries I get a message trying to update the bibtex file:

    “Zotero record has been changed. Press OK tu run ‘Update Bibtex’and insert the citation…”
    But firefox needed a few minutes, so perhaps with a larger database that might be what causes firefox to freeze….
    However hitting the “OK”-Button will freeze firefox again…
    Now on the fresh Ubuntu profile I even can’t add one entry, irrespectively if I use a new Lyx file or a file already known by Lyz. Sooner or later Lyz seems to crash.

    Thanks again for Lyz, without it I would have to use OpenOffice 😉 I hope I don’t cause you too much trouble :) Bye

  • john

    short update: now with my first/main zotero database (after rebooting from windows where I checked if Lyz in Windows works) I was able to add at least 10 entries to Lyx via Lyz in Ubuntu. So the Problem might not be related to Ubuntu alone. Unfortunately updating the bibtex file after adding some entries, Lyz at first showed all the entries I added in Windows to tell me that these will be added to the bibtex file (even though they are still in there as Lyz in Windows added them), but after hitting “OK” Firefox is frozen again…

  • Petr

    John, thanks for the input.
    I have actually already realized the absolute path problem, as I also use both linux and windows, but lately I had little time for it. I promise to look at it during next week.
    As for the lyx server, all the freezing is caused by miscommunication with lyxserver, I will have to investigate more. I have noticed occasional problems earlier, which seemed to go away with a little timeout added. Sometimes the server seems to be little lazy. I’ll try to find out soon and correct all this behaviour.
    I personally use the ‘update’ rarely usually just for one or two slightly modified entries. For what purpose do you typically use it?

  • john

    Petr,
    thanks for your fast answer. I am glad to hear that you use linux and windows, too. and that therefore you also think about switching to relative path names :)
    I think that the lazy Lyx server might be the problem, too, as the faster you add entries via Lyz to Lyx the earlier you will run into trouble.
    At the moment I don’t use Lyx and Lyz productively, I am just doing some tests to find out if it fits my need to write 200 pages 😉 . Therefore I also tested the “update bibtex” function a lot. I really like this feature as I have the ability to change my entries in Zotero without loosing all entries in Lyx. Therefore this feature becomes more and more important the more I have already cited. So I think i will use this feature regularly as I notice failures in my references often really lately 😉
    I will be glad to do some tests with the new Lyz version in a few weeks :)

  • Petr

    Great. You’re on! 😉 I am actually bold enough to write my thesis with it and so far I didn’t have to curse… There are always the backup files before each update… I’ll post the totally bug-free revision soon 😉

  • john

    Petr,

    if you like to, you can gladly send me some alpha/beta/pre-release versions via email. I can do some user-like testing, e.g. migrating an already existing lyx and bibtex file from absolute to relative paths etc.

  • Petr

    Thanks, will do :)

  • Simon

    Hi. Thank you for the Lyz-plugin. I have been following the discussion here for the last weeks, since I have started writing my thesis (in theology) in Lyx. Sadly I can not get the piping to work (maybe because I am using Zotero 2.0.3). But I am really looking forward to a bug-free version soon:-) (By now I can’t get my bibliography into my document; all the special characters are strange when I just save it as a .bib-file). Sorry for this long message and thank you again for your work

  • Reinier

    Dear Petr and others,
    I am a novice with lyx and zotero. I followed all the steps to get it work but failed. Zotero is normally installed and Lyz too. If i want to citate in lyx, i get an error: “Wrong path to Lyx server. Set the path specified in Lyx preferences.” and “Could not contact server at: \\.\pipe\lyxpipe” and “Could not retrieve document name.”

    I really don’t have a clue what i’ve done wrong. I hope you can help. I suppose it’s a lyx problem?

    • Petr

      Reiner,
      I assume you are using windows. Make sure that “\\.\pipe\lyxpipe” is set in Lyx. Naturally, Lyx has to be running. You can start lyx by “lyx.exe -dbg lyxserver” from commandline to see if it’s really working.
      Cheers
      Petr

  • Tom

    With Firefox 3.6.3 and Lyz 1.6.7.1 on Mac OS the update Bibtex function does not seem to work. I don’t get any error message, but it is not updating the Bibtex file…

  • Petr

    Might be related to a lyxserver problem. I will try to write up an alternative communication with the lyxserver soon (couple of days, as I am too busy now). In case you speak javascript, don’t hesitate to have a look at the source code at launchpad 😉

  • Tom

    Ahhh, I think it’s because I deleted a reference in Zotero…

  • Petr

    Right, that’s possible. This is already fixed for the next release. Give me few days.

  • Pol

    Hi Petr,

    I’m really impressed by the functionality. I was just looking at switching back to Lyx from OpenOffice but the key point for me was the Zotero integration as I got used to it very much. Unfortunately I admit I encounter a lot of LyxServer communication issues (Firefox freezes), so to get 3-4 citations through it may even require 10 restarts of Lyx and Firefox. I’m on Kubuntu 10.04 with disabled KDE integration in Firefox (I noticed it caused trouble). Apart from standard Kubuntu versions (FF 3.6.3, Zotero 2.0.3, Lyx 1.6.5) I tried to test it with a freshly compiled Lyx (1.6.6.1) but the rate of crashing is similar. I’d be happy to test any bug-fixed versions so please send me any newer ones.
    BTW, how do you create a FF plugin file from the sources downloaded using bzr from Launchpad?

    • Petr

      Fix is basically ready, I just had little time lately to finish it up. It will appear on FF addons soon.
      FF plugins are simple zip files with extension .xpi. I have a Makefile for it:
      all:
      rm -rf lyz.xpi && zip -r lyz.xpi ./*
      Petr

  • John

    Hi Pol,

    I had the same as you encountered so I just wanted to say thx to Petr here for fixing this bug. I am glad that also other people (and not only me) can benefit from the new version :)

  • Michael

    Hi Petr,
    thank you for the Lyz-plugin – it’s doing a great job! Nevertheless I noticed a little glitch on Mac OS X 10.6.4 using Lyx 1.6.5, Firefox 3.6.6 and Lyz 1.6.8.1.
    When I change an existing Zotero item (e.g. author name) and then use “Update BibTeX” from the Lyz menu I get the error “Wrong path to Lyx server. Set the path specified in Lyx preferences” before Lyz asks me if I want to update the lyx document. None the less the lyx document will be updated correctly after I close and reopen it manually.
    I do not know if I am the only one witnessing this behavior but it’s truely just a minor inconvenience.
    I Hope, I could help you to make Lyz even better 😉
    Michael

    • Petr

      Hi Michael,
      you’re not the only one. It seems to be happening from time to time, for reasons unknown, so far that is :)
      Occasionally I am trying to find out what’s going on, but being too busy recently, I haven’t gave it too much effort (I’m hoping someone comes by, looks at the code and finds the problem ;)).
      Take care
      Petr

Leave a Reply