I am currently playing with floccus to store bookmarks using Nextcloud, so I have created simple shell scripts to import Nextcloud bookmarks from Firefox using code from my earlier solution to open Firefox bookmarks from OpenBox menu.
Prerequisites
Install curl utility to perform API calls.
$ sudo apt-get install curl
Install jq utility to parse JSON files.
$ sudo apt-get install jq
Install sqlite3 utility to access SQLite database.
$ sudo apt-get install sqlite3
Create application password
Create Nextcloud application password, do not use your regular credentials.
Bookmarks
I will use the following bookmarks in this article. These bookmarks are included in source code.
Mozilla Firefox>Pomoc (https://support.mozilla.org/pl/products/firefox) Mozilla Firefox>Dostosuj Firefoksa (https://www.mozilla.org/pl/firefox/customize/) Mozilla Firefox>Dołącz do nas (https://www.mozilla.org/pl/contribute/) Mozilla Firefox>O Mozilli (https://www.mozilla.org/pl/about/) Ubuntu and Free Software links>Ubuntu (http://www.ubuntu.com/) Ubuntu and Free Software links>Ubuntu Wiki (community-edited website) (http://wiki.ubuntu.com/) Ubuntu and Free Software links>Make a Support Request to the Ubuntu Community (https://answers.launchpad.net/ubuntu/+addquestion) Ubuntu and Free Software links>Debian (Ubuntu is based on Debian) (http://www.debian.org/) Apps>The most popular self-hosted file share and collaboration platform (https://nextcloud.com/) Apps>Open Source Password Management Solutions | Bitwarden (https://bitwarden.com/) News>Welcome to LWN.net [LWN.net] (https://lwn.net/) News>Services>NewsBlur (https://newsblur.com/) Personal>Services>sleeplessbeastie's notes (https://blog.sleeplessbeastie.eu/) Debian>Debian -- The Universal Operating System (https://www.debian.org/) Debian>Planet Debian (https://planet.debian.org/)
Display and export bookmarks
Create nextcloud_bookmarks_print.sh shell script.
Shell script
#!/bin/sh
# Export Firefox bookmarks
# path to the sqlite3 binary
sqlite_path=$(which sqlite3)
# sqlite3 parameters (define separator character)
sqlite_params="-separator ^"
# path to the places.sqlite database
bookmarks_database=$(ls ~/.mozilla/firefox/*.default/places.sqlite)
# SQL query
sql_query="select p.title, p.url from moz_places as p where p.hidden=0 order by last_visit_date desc limit 10"
# root element
root_element_query="select id from moz_bookmarks where rtrim(guid,'_')='menu'"
root_element="$($sqlite_path $sqlite_params "$bookmarks_database""$root_element_query")"
# process bookmarks
process_bookmarks(){
# SQL query - folders
folder_id=$1
folder_path=""
while [ "$folder_id" != "2" ]; do
sql_folder_query="select parent,title from moz_bookmarks where id=$folder_id and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"
sql_folder_result=$($sqlite_path $sqlite_params "$bookmarks_database""$sql_folder_query" )
folder_id=$(echo $sql_folder_result | awk -F^ '{print $1}')
folder_title=$(echo $sql_folder_result | awk -F^ '{print $2}')
# special case for empty title
if [ -z "$folder_path" ]; then
folder_path="floccus:>$folder_title"
else
folder_path="${folder_path}>${folder_title}"
fi
done
# escape special characters
folder_path=$(echo $folder_path | sed -e "s/&/\&/g" -e "s/\"/\"/g" -e "s/</\</g" -e "s/>/\>/g")
# SQL query - bookmarks
sql_bookmarks_query="select b.title, p.url from moz_bookmarks as b left outer join moz_places as p on b.fk=p.id where b.type = 1 and p.hidden=0 and b.title not null and parent=$1"
$sqlite_path $sqlite_params "$bookmarks_database""$sql_bookmarks_query" | while IFS=^ read title url; do
# special case for empty title
if [ -z "$title" ]; then
title=$url
fi
# escape special characters
title=$(echo $title | sed -e "s/&/\&/g" -e "s/\"/\"/g" -e "s/</\</g" -e "s/>/\>/g")
url=$(echo $url | sed -e "s/&/\&/g" -e "s/\"/\"/g" -e "s/</\</g" -e "s/>/\>/g")
e display url, title, path
echo "<DT><A HREF=\"${url}\" TAGS=\"${folder_path}\">${title}</A>"
done
}
# process the folders function
process_folders(){
# execute only when there is an axactly one parameter
if [ "$#" = 1 ]; then
# SQL query - folders
sql_folder_query="select id from moz_bookmarks where parent=$1 and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"
# process folders
$sqlite_path $sqlite_params "$bookmarks_database""$sql_folder_query" | while IFS=^ read id; do
# process folders inside
process_folders $id
# process bookmarks in current folder
process_bookmarks $id
done
fi
}
# header
echo "<!DOCTYPE NETSCAPE-Bookmark-file-1>"
echo "<META HTTP-EQUIV=\"Content-Type\" CONTENT=\"text/html; charset=UTF-8\">"
echo "<!-- This is an automatically generated file."
echo "It will be read and overwritten."
echo "Do Not Edit! -->"
echo "<TITLE>Bookmarks</TITLE>"
echo "<H1>Bookmarks</H1>"
echo "<DL><p>"
# process folders
process_folders "$root_element"
# process bookmarks for root element
process_bookmarks "$root_element"Usage
Export Firefox bookmarks to specified filename using floccus tags to preserve path.
$ nextcloud_bookmarks_print.sh | tee export.html <!DOCTYPE NETSCAPE-Bookmark-file-1> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8"> <!-- This is an automatically generated file. It will be read and overwritten. Do Not Edit! --> <TITLE>Bookmarks</TITLE> <H1>Bookmarks</H1> <DL><p> <DT><A HREF="https://support.mozilla.org/pl/products/firefox" TAGS="floccus:>Mozilla Firefox">Pomoc</A> <DT><A HREF="https://www.mozilla.org/pl/firefox/customize/" TAGS="floccus:>Mozilla Firefox">Dostosuj Firefoksa</A> <DT><A HREF="https://www.mozilla.org/pl/contribute/" TAGS="floccus:>Mozilla Firefox">Dołącz do nas</A> <DT><A HREF="https://www.mozilla.org/pl/about/" TAGS="floccus:>Mozilla Firefox">O Mozilli</A> <DT><A HREF="http://www.ubuntu.com/" TAGS="floccus:>Ubuntu and Free Software links">Ubuntu</A> <DT><A HREF="http://wiki.ubuntu.com/" TAGS="floccus:>Ubuntu and Free Software links">Ubuntu Wiki (community-edited website)</A> <DT><A HREF="https://answers.launchpad.net/ubuntu/+addquestion" TAGS="floccus:>Ubuntu and Free Software links">Make a Support Request to the Ubuntu Community</A> <DT><A HREF="http://www.debian.org/" TAGS="floccus:>Ubuntu and Free Software links">Debian (Ubuntu is based on Debian)</A> <DT><A HREF="https://nextcloud.com/" TAGS="floccus:>Apps">The most popular self-hosted file share and collaboration platform</A> <DT><A HREF="https://bitwarden.com/" TAGS="floccus:>Apps">Open Source Password Management Solutions | Bitwarden</A> <DT><A HREF="https://lwn.net/" TAGS="floccus:>News">Welcome to LWN.net [LWN.net]</A> <DT><A HREF="https://newsblur.com/" TAGS="floccus:>News>Services">NewsBlur</A> <DT><A HREF="https://blog.sleeplessbeastie.eu/" TAGS="floccus:>Personal>Services">sleeplessbeastie's notes</A> <DT><A HREF="https://www.debian.org/" TAGS="floccus:>Debian">Debian -- The Universal Operating System</A> <DT><A HREF="https://planet.debian.org/" TAGS="floccus:>Debian">Planet Debian</A>
You can import created file to Nextcloud Bookmarks using web-interface or restore Nextcloud bookmarks shell script.
Use Nextcloud API to import bookmarks from Firefox
This is an automated solution, that will use API to import Firefox bookmarks directly to Nextcloud bookmarks application. Create firefox_bookmarks_add.sh shell script.
Shell script
#!/bin/sh
# Import Firefox bookmarks to Nexcloud
# https://blog.sleeplessbeastie.eu/
# path to the sqlite3 binary
sqlite_path=$(which sqlite3)
# sqlite3 parameters (define separator character)
sqlite_params="-separator ^"
# path to the places.sqlite database
bookmarks_database=$(ls ~/.mozilla/firefox/*.default/places.sqlite)
# SQL query
sql_query="select p.title, p.url from moz_places as p where p.hidden=0 order by last_visit_date desc limit 10"
# root element
root_element_query="select id from moz_bookmarks where rtrim(guid,'_')='menu'"
root_element="$($sqlite_path $sqlite_params "$bookmarks_database""$root_element_query")"
# escape html
escape_html() {
echo $* | perl -n -mHTML::Entities -e "print HTML::Entities::encode_entities_numeric(\$_,'<>&\"\'[](){}#@|%+')"
}
# process bookmarks
process_bookmarks(){
# create folder path
folder_id=$1
folder_path=""
while [ "$folder_id" != "2" ]; do
sql_folder_query="select parent,title from moz_bookmarks where id=$folder_id and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"
sql_folder_result=$($sqlite_path $sqlite_params "$bookmarks_database""$sql_folder_query" )
folder_id=$(echo $sql_folder_result | awk -F^ '{print $1}')
folder_title=$(echo $sql_folder_result | awk -F^ '{print $2}')
# special case for empty title
if [ -z "$folder_path" ]; then
folder_path="floccus:>$folder_title"
else
folder_path="${folder_path}>${folder_title}"
fi
done
# process bookmarks
sql_bookmarks_query="select b.title, p.url from moz_bookmarks as b left outer join moz_places as p on b.fk=p.id where b.type = 1 and p.hidden=0 and b.title not null and parent=$1"
$sqlite_path $sqlite_params "$bookmarks_database""$sql_bookmarks_query" | while IFS=^ read ff_title ff_url; do
# check if url is already stored before adding it
found_bookmark="0"
continue_pagination="1"
page=0
while [ "${continue_pagination}" -eq "1" ]; do
urls=$(curl --silent -X GET --user "${param_username}:${param_password}" \
--header "Accept: application/json" \
"${param_nextcloud_address}/index.php/apps/bookmarks/public/rest/v2/bookmark?page=${page}&search\[\]=$(escape_html $ff_url)" | \
jq -r '.data[].url')
if [ -z "${urls}" ]; then
continue_pagination="0"
else
for url in $urls; do
if [ "$ff_url" == "$url" ]; then
found_bookmark="1"
break
fi
done
fi
if [ "${found_bookmark}" -eq "0" ]; then
status=$(curl --silent -X POST --user "${param_username}:${param_password}" \
--data-urlencode "url=$ff_url" \
--data-urlencode "title=$ff_title" \
--data-urlencode "item[tags][]=$folder_path" \
"${param_nextcloud_address}/index.php/apps/bookmarks/public/rest/v2/bookmark" | \
jq -r 'select(.status != "success") | .status')
if [ -n "${status}" ]; then
echo "Skipped Nextcloud bookmark url \"${ff_url}\" with title \"${ff_title}\" and tag \"${folder_path}\"."
else
echo "Added Nextcloud bookmark url \"${ff_url}\" with title \"${ff_title}\" and tag \"${folder_path}\"."
fi
continue_pagination="0"
page="0"
else
page=$(expr $page + 1)
fi
done
done
}
# process the folders function
process_folders(){
# execute only when there is an exactly one parameter
if [ "$#" = 1 ]; then
# SQL query - folders
sql_folder_query="select id from moz_bookmarks where parent=$1 and type=2 and (select count(*) from moz_bookmarks as b2 where b2.parent=moz_bookmarks.id)>0"
# process folders
$sqlite_path $sqlite_params "$bookmarks_database""$sql_folder_query" | while IFS=^ read id; do
# process folders inside
process_folders $id
# process bookmarks in current folder
process_bookmarks $id
done
fi
}
# usage info
usage(){
echo "Usage:"
echo " $0 -r nextcloud_url -u username -p passsword"
echo ""
echo "Parameters:"
echo " -r nextcloud_url : set Nextcloud URL (required)"
echo " -u username : set username (required)"
echo " -p password : set password (required)"
echo ""
}
# parse parameters
while getopts "r:u:p:" option; do
case $option in
"r")
param_nextcloud_address="${OPTARG}"
param_nextcloud_address_defined=true
;;
"u")
param_username="${OPTARG}"
param_username_defined=true
;;
"p")
param_password="${OPTARG}"
param_password_defined=true
;;
\?|:|*)
usage
exit
;;
esac
done
if [ "${param_nextcloud_address_defined}" = true ] && \
[ "${param_username_defined}" = true ] && \
[ "${param_password_defined}" = true ]; then
# process folders
process_folders "$root_element"
# process bookmarks for root element
process_bookmarks "$root_element"
else
usage
fi
Usage
Display usage information.
$ firefox_bookmarks_add.sh Usage: firefox_bookmarks_add.sh -r nextcloud_url -u username -p passsword Parameters: -r nextcloud_url : set Nextcloud URL (required) -u username : set username (required) -p password : set password (required)
Import Firefox bookmarks.
$ firefox_bookmarks_add.sh -r https://cloud.example.org/ -u milosz -p Mjdu3-kDnru-4UksA-fYs0w Added Nextcloud bookmark url "https://support.mozilla.org/pl/products/firefox" with title "Pomoc" and tag "floccus:>Mozilla Firefox". Added Nextcloud bookmark url "https://www.mozilla.org/pl/firefox/customize/" with title "Dostosuj Firefoksa" and tag "floccus:>Mozilla Firefox". Added Nextcloud bookmark url "https://www.mozilla.org/pl/contribute/" with title "Dołącz do nas" and tag "floccus:>Mozilla Firefox". Added Nextcloud bookmark url "https://www.mozilla.org/pl/about/" with title "O Mozilli" and tag "floccus:>Mozilla Firefox". Added Nextcloud bookmark url "http://www.ubuntu.com/" with title "Ubuntu" and tag "floccus:>Ubuntu and Free Software links". Added Nextcloud bookmark url "http://wiki.ubuntu.com/" with title "Ubuntu Wiki (community-edited website)" and tag "floccus:>Ubuntu and Free Software links". Added Nextcloud bookmark url "https://answers.launchpad.net/ubuntu/+addquestion" with title "Make a Support Request to the Ubuntu Community" and tag "floccus:>Ubuntu and Free Software links". Added Nextcloud bookmark url "http://www.debian.org/" with title "Debian (Ubuntu is based on Debian)" and tag "floccus:>Ubuntu and Free Software links". Added Nextcloud bookmark url "https://nextcloud.com/" with title "The most popular self-hosted file share and collaboration platform" and tag "floccus:>Apps". Added Nextcloud bookmark url "https://bitwarden.com/" with title "Open Source Password Management Solutions | Bitwarden" and tag "floccus:>Apps". Added Nextcloud bookmark url "https://lwn.net/" with title "Welcome to LWN.net [LWN.net]" and tag "floccus:>News". Added Nextcloud bookmark url "https://newsblur.com/" with title "NewsBlur" and tag "floccus:>News>Services". Added Nextcloud bookmark url "https://blog.sleeplessbeastie.eu/" with title "sleeplessbeastie's notes" and tag "floccus:>Personal>Services". Added Nextcloud bookmark url "https://www.debian.org/" with title "Debian -- The Universal Operating System" and tag "floccus:>Debian". Added Nextcloud bookmark url "https://planet.debian.org/" with title "Planet Debian" and tag "floccus:>Debian".
Additional notes
Download source code, Firefox bookmarks exported in json and html format are included too.
Remember to backup Nextcloud bookmarks.
