Node.js based ports : how do you handle the npm install?

H

hukadan

Guest
Hello,

I have been trying to make a port for NodeBB, a forum software using Node.js. Once the project has been fetched, you also need to install dependencies (a lot of them) using the npm install command. But when building with poudriere(8), you have access to the network only during (pre/post-)fetch phase. And so, npm install command will fail.

As a workaround, I did a really dirty hack by creating a directory in distfiles and then ran npm install during a post-fetch phase while the network is still avaible. At the end, the port build and install just fine. But this solution looks really ugly to me.

Have you got any idea about how this could be handled in a better way ? Of course I tried to look to other Node.js based ports but the few I saw had the same issue (https://svnweb.freebsd.org/ports/head/misc/teslams/Makefile?view=markup) or no dependencies.

Here is the Makefile (that's my first port attempt so all comments/advises are welcome) :

Code: 
# Created by: Hukadan <hukadan@xxxxx>
# $FreeBSD$

PORTNAME= nodebb
PORTVERSION= 1.0.3
DISTVERSIONPREFIX= v
CATEGORIES= www

MAINTAINER= hukadan@xxxxxxxx
COMMENT= Node.js based bulletin board / discussion forum system

LICENSE= GPLv3

FETCH_DEPENDS= npm>=3.0.0:www/npm \
               git:devel/git
RUN_DEPENDS= npm>=3.0.0:www/npm

USERS= _pma
NODEBB_USERNAME= ${USERS}

WRKSRC= ${WRKDIR}/NodeBB-${PORTVERSION}
NPMDIR= ${DISTDIR}/npm-trick
NO_ARCH= yes
NO_BUILD= yes
PLIST_SUB= NODEBB_USERNAME=${NODEBB_USERNAME}

USE_GITHUB= yes
GH_ACCOUNT= nodebb

USE_RC_SUBR= nodebb

USES= shebangfix
SHEBANG_FILES= nodebb
SHEBANG_LANG= node
node_OLD_CMD= "/usr/bin/env node"
node_CMD= ${LOCALBASE}/bin/node

post-fetch:
    @${MKDIR} ${NPMDIR}
    @cd ${NPMDIR} && ${TAR} -xf ${DISTDIR}/${DISTNAME}${EXTRACT_SUFX}
    @cd ${NPMDIR}/NodeBB-${PORTVERSION} && ${SETENV} npm install

do-install:
    @${MKDIR} ${STAGEDIR}${WWWDIR}
    @cd ${NPMDIR}/NodeBB-${PORTVERSION} && ${COPYTREE_SHARE} . ${STAGEDIR}${WWWDIR}

post-install:
    @${CHMOD} +x ${STAGEDIR}${WWWDIR}/${PORTNAME}
    @${RM} -Rf ${NPMDIR}

.include <bsd.port.mk>

Thank you.
 
I know this is probably not what you want to hear as you've been trying to make it into a port... but nodeBB is itself a node module, so in theory you should be able to just npm install nodeBB, or add it as a dependency to your nodeBB based project. If you want to continue down the port route then i think your question is more port specific so i'll let the experts answer the networking issue.

On a side note, if you are installing it as a port then it's sort of a global thing, in which case npm -g would be your equivalent.
 
Thank you both for your replies.

tomxor, I would like to install NodeBB the same way www/owncloud is installed, in the /usr/local/www/ directory since it will provide a web service, namely a forum. I also want the file permissions to be set during the installation, since the process running the nodebb service should have write access to only some of them. Once again, I took example with www/owncloud. This can be handled easily with pkg-plist. For exemple, some of these advice are implemented with the following lines in the pkg-plist :
Code: 
...
@dir(%%OWNCLOUD_USERNAME%%,%%OWNCLOUD_GROUPNAME%%,) %%WWWDIR%%/apps
@dir(%%OWNCLOUD_USERNAME%%,%%OWNCLOUD_GROUPNAME%%,) %%WWWDIR%%/config
@dir(%%OWNCLOUD_USERNAME%%,%%OWNCLOUD_GROUPNAME%%,) %%WWWDIR%%/data
I also want a rc(8) script to be installed. All this cannot be achieved using npm install.

SirDice, the npm install not only install the given package but also fetches dependencies. As far as I can tell, www/uglifyjs has no dependency (from an npm point of view) and therefore does not need access to the network when doing npm install. On the contrary, NodeBB has dependencies that are fetched during the npm install.

At the moment, I am looking for a way to provide an archive that bundle all the dependencies which could be fetched and then installed offline. I think this might be a better way to achieve what I want in a clean way. I will post my findings here (if any).
 
hukadan said:
At the moment, I am looking for a way to provide an archive that bundle all the dependencies which could be fetched and then installed offline. I think this might be a better way to achieve what I want in a clean way. I will post my findings here (if any).
Click to expand...

I think it would be worthwhile finding a way for pkg() to (successfully) invoke the appropriate npm() commands for all ports who's dependencies live in the nodeJS world.

One problem with taking a snapshot of the node_modules dependency tree is you wouldn't get a most up to date version of that tree every time the port is installed or updated, the trees can be very wide, deep and modules very granular, so there is a lot of pieces to keep up to date, the way that dependencies are defined in node allow them to be specified implicitly with semver... so while the main package might not have changed, it's dependencies could.

Considering this is a forum - security could matter and the modules can potentially be implicated in security (of the forums DB at least), so allowing pkg install and pkg update to invoke the npm() equivalents would be a lighter solution that also takes the responsibility of dependency bugs off the port maintainer.
 
tomxor, thank you for your advice.

tomxor said:
I think it would be worthwhile finding a way for pkg() to (successfully) invoke the appropriate npm() commands for all ports who's dependencies live in the nodeJS world.
Click to expand...
I think it would be complicated and error prone.

tomxor said:
One problem with taking a snapshot of the node_modules dependency tree is you wouldn't get a most up to date version of that tree every time the port is installed or updated, the trees can be very wide, deep and modules very granular, so there is a lot of pieces to keep up to date, the way that dependencies are defined in node allow them to be specified implicitly with semver... so while the main package might not have changed, it's dependencies could.

Considering this is a forum - security could matter and the modules can potentially be implicated in security (of the forums DB at least), so allowing pkg install and pkg update to invoke the npm() equivalents would be a lighter solution that also takes the responsibility of dependency bugs off the port maintainer.
Click to expand...

This is my main concern. In my experimentation, I keep the npm-shrinkwrap.json and package.json files for each node_modules tree. Then I can use nsp (a king of pkg-audit(8) for Node.js modules) to check for known vulnerabilities. This could be automated to send an e-mail to the port maintainer when such a vulnerability has been found for a given node_modules tree. I am not saying this is the way to go, but it is what I am trying at the moment.
 
You must log in or register to reply here.
Back
Top