LyX ⇒ Biber cannot find <filename>.bib file with OneDrive

[Gnosos]

Hi,

I am running LyX 2.2.2 on MacOS 10.12.2 with MacTeX 2016basic installed (plus a few packages added via TeXLive). The document I am working on has several sections, so a parent folder holds common files and a master document, while each section of the document is stored in a subfolder as a child document. Because I work from several different computers, both the parent and child folders are themselves descendants in a OneDrive (Version 17.3.6725) folder, which takes care of synchronization between machines.

I want to be able to compose and read each section of the document by itself. So each one is set up as an independent document, but things they share in common are stored in the parent folder. For example, two .tex configuration files are stored in the parent, so that the LaTeX preamble of each child file just has to have a \input{../<configuration file name>} command for each of the two files.

The problem I'm having is with the .bib file. It's named "References.bib." Because it's shared by each section of the document, it resides in the parent folder. When I compile a LyX file located in the parent folder, everything works fine. But when I try to compile one of the sections stored in a subfolder, I get an error message, "Biber error: Cannot find 'References.bib'!" with a descriptition, "[281] Utils.pm:180> ERROR - Cannot find 'References.bib'!"

I suspect the problem is how the path to References.bib is being set up in LyX by using Insert > List/TOC > BibTeX Bibliography. When the LyX file resides in the parent folder, LyX sets the database to just "References." But when it's a LyX file for one of the sections and resides in one of the child folders, LyX takes a very long time; returns with the full, absolute path, /Users/..../<parent folder>/References.bib, which only appears at first in the Lyx: "Add BibTeX database" window that pops up when one clicks the Add button (along with things like "biblatex-examples"; and then it can be selected by pressing Add in the Databases window, selecting it from the list in the "Add BibTeX database" window, and then pressing Add > OK. The only times References.bib cannot be found are when LyX is set up this way for a file in one of the child folders.

Maybe the problem is the extraordinarily long time LyX takes to set up the path to the database is also an issue during the compile. If the LaTex system is looking for References.bib, but this takes a long time, perhaps the system has some sort of timing mechanism that waits for a certain period and then decides the file cannot be found. Or, perhaps it has something to do with how OneDrive actually implements files.

I also suspected the problem might be solved if, instead of browsing to the References.bib file, I just entered a relative path: ../References or ../References.bib. But I tried each of these without success. I also tried making a symbolic link to References.bib, moving it into the child folder, and inserting this link into the LyX file in the folder. But this has similar results in that References.bib is not found.

So Help! How can I keep a .bib file in a parent directory on OneDrive and use the file from a LyX file in one of the parent directory's child folders?

[Gnosos]

Following up my earlier post, I decided to problem-solve by isolating different issues. So I created a simple article-class document in the regular, local file system and made a small References.bib file in the same folder. I also ran Disk Utility several times and uninstalled LyX then reinstalled it.

Even with this simple, relatively standard setup, inserting the Bibliography BibTeX link is taking an inordinate amount of time, freezes the LyX app, and the only way to recover is to Force Quit it. This only happens if one tries to browse for the .bib file. If one simply types it in, everything works fine. Still, the browse function seems broken.

[Gnosos]

Well after considerable additional testing & work, I finally got this to work as desired. The main point for posterity is that there seems to be a bug in LyX. If one uses Insert > List/TOC > BibTeX Bibliography > Add > Browse, LyX hangs and freezes. I timed it, and LyX was frozen from 10:35 to 11:55, at which time I Force Quit it.

To make matters worse, clicking the Open button in the Browse dialog window, returns you to the previous, Add window. When LyX freezes, nothing in the window actually works, although some of the controls appear to be still active. Before narrowing down the problem, a few times when I tested this, LyX eventually did add the file selected by Browsing. But looking at the Add window, there's no certain way to know that LyX has thawed. You may see your selected file appear in the list of .bib file candidates. Or you may not, in which case you'd have to Cancel to back up to the previous window (BibTeX Bibliography), and then try Add again, at which time you may see the file you selected via Browse. But while the Add window is frozen, all you can do is keep trying because there's no feedback to indicate the browse is still working to retrieve the path to your bibliography file. So this can be a great time waster. This appears to be a rather serious bug.

Because this seemed related to the state of the filesystem, I mistakenly associated it with the fact I was using OneDrive. Since I was able to reproduce the issue on the local filesystem, I am now convinced that OneDrive is innocent.

But here's a workaround for the problem:

1. Locate your .bib file in the parent directory.
2. Create a symbolic link to the .bib file. DO NOT create an alias link, as aliases don't work.
   • The easiest way I found to create a symbolic link is to open a Terminal window and navigate to the parent folder. Then issue the command: y=`pwd`. (The exact syntax depends on what shell you're using. I believe the default shell is bash, and this version of the command works.) What this will do is to store the path to the parent in shell variable y.
   • Next navigate to the child folder and issue the command: ln -s "$y/foo.bib" foo.bib. Here "foo.bib" is the name of your .bib file. This command will create a symbolic link to the original file in the parent folder and name it identically (except for the path).
   • If you are using other child folders, you can simply copy and paste this symbolic link into them.
3. Next, in LyX set up your child documents to access the .bib file through the symbolic link. Using BibLaTeX and LyX 2.2.2, follow the instructions here. You'll want to add variants of the following to your preamble:
   • \usepackage[style=authoryear,backend=biber]{biblatex}. Personally I use \usepackage[authordate,natbib,backend=biber]{biblatex-chicago} because Chicago A-D is most common in my world.
   • \addbibresource{foo.bib}
     I also suggest you follow Aaron DeFazio's suggestions for organizing complex papers being written in LyX. The main innovation is having two configuration files in the parent folder and then accessing them in each child document's preamble as \input{../general-preamble} and \input{../master-preamble}. The two commands above go into the general-preamble.
4. Insert a \bibliography statement in your child document, as this will allow you to use LyX's citation feature. But in light of the above, do it as follows:
   • Insert > List/TOC > BibTeX Bibliography > Add
   • But instead of browsing to your .bib file, just type its name in the text box on the Add window and press Add, OK to back out.
   • For some reason, LyX will eventually lose track of this, and you won't see your .bib file listed in the list of databases in the BibTeX Bibliography window. Nonetheless, once you set up the \bibliography command and save your file, LyX will continue to use it. Thankfully, the command is only used by LyX to enable its citation feature, so there's not much damage done except the name disappears from the list.
5. Add a \printbibliography command in ERT where you want your bibliography to appear. I use \printbibliography[title={References}] to name the section.
6. Wrap the ERT block containing the \printbibliography command in a branch. Then you can activate the branch when you want to compile just the child document and deactivate it when you want to include it in the master file in the parent folder.
7. I also found it necessary to use Document > Settings ... > Modules to add Biblatex-citation-styles to the child document.

Have fun!

[scottkosty]

Thanks for taking the time to write down the details of the workaround that worked for you! I tried on LyX 2.2.2 and I cannot reproduce this problem on Ubuntu, so I wonder if it is something OS-specific. I wonder if it is related to http://www.lyx.org/trac/ticket/10396