Pike on OpenBSD

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Pike on OpenBSD

Bernte
Hi -

I would love to run Roxen on my OpenBSD server, and the first step is to
get a current version of Pike to work on OpenBSD.

I have installed the following packages:

libnettle-3.2
gmp-5.0.2p3
gmake-4.2.1

The first hurdle is compilation of efuns.c:

Compiling modules/_Stdio/efuns.c
/home/schoelle/Code/Pike-v8.0.370/src/modules/_Stdio/efuns.c: In
function 'f_get_dir':
/home/schoelle/Code/Pike-v8.0.370/src/modules/_Stdio/efuns.c:1402:
error: 'name_max' undeclared (first use in this function)
/home/schoelle/Code/Pike-v8.0.370/src/modules/_Stdio/efuns.c:1402:
error: (Each undeclared identifier is reported only once
/home/schoelle/Code/Pike-v8.0.370/src/modules/_Stdio/efuns.c:1402:
error: for each function it appears in.)

The corresponding code is:

void f_get_dir(INT32 args)
{
[...]
#ifdef HAVE_READDIR_R
   ptrdiff_t name_max = -1;
#endif

[...]

   low_get_dir(dir, name_max);
   stack_pop_n_elems_keep_top(args);
}
#endif /* __NT__ */

So name_max is defined conditionally, but it is used unconditionally.

Removing the #ifdef, compilation continues, but then I get the next error:

Compiling post_modules/GL/top.c
/home/schoelle/Code/Pike-v8.0.370/build/openbsd-6.0-amd64/tpike
-DNOT_INSTALLED -DPRECOMPILED_SEARCH_MORE
-m/home/schoelle/Code/Pike-v8.0.370/build/openbsd-6.0-amd64/master.pike
/home/schoelle/Code/Pike-v8.0.370/src/post_modules/GL/gen.pike
/home/schoelle/Code/Pike-v8.0.370/src/post_modules/GL/auto.c.in > auto.c.tmp
tpike(85013) in free(): error: bogus pointer (double free?) 0xebdd6208440
Abort trap (core dumped)

A stacktrace in GDB yields:

#0  0x00000c4a6ff37bca in thrkill () at <stdin>:2
#1  0x00000c4a6ff449d9 in *_libc_abort () at
/usr/src/lib/libc/stdlib/abort.c:52
#2  0x00000c4a6ff64309 in wrterror (d=0xc4a8a8c1270,
     msg=0xc4a700b8dc3 "bogus pointer (double free?)", p=0xc47fd308440)
     at /usr/src/lib/libc/stdlib/malloc.c:295
#3  0x00000c4a6ff657db in free (ptr=0xc47fd308440) at
/usr/src/lib/libc/stdlib/malloc.c:1364
#4  0x00000c47fcbd02b9 in really_free_mapping (m=0xc4ae765f0f0)
     at /home/schoelle/Code/Pike-v8.0.370/src/mapping.c:307
#5  0x00000c47fcbdf02d in destruct_object (o=0xc4a86d9f340,
reason=Variable "reason" is not available.
)
     at /home/schoelle/Code/Pike-v8.0.370/src/object.c:972
#6  0x00000c47fcbe16fc in destruct_objects_to_destruct_cb ()
     at /home/schoelle/Code/Pike-v8.0.370/src/object.c:1067
#7  0x00000c47fcb428d3 in eval_instruction (
     pc=0xc4a9dbb065e
"\006X\001�\001X\002�\001X\003�\001X\004�\001X\005�\023�8X\006\216")
     at interpret_functions.h:2718
#8  0x00000c47fcb47310 in mega_apply_low (args=Variable "args" is not
available.
)
     at /home/schoelle/Code/Pike-v8.0.370/src/interpret.c:2721
#9  0x00000c47fcbde3ce in call_pike_initializers (o=0xc4a86d9f0d0, args=0)
     at /home/schoelle/Code/Pike-v8.0.370/src/object.c:338
#10 0x00000c47fcbe25af in get_master () at
/home/schoelle/Code/Pike-v8.0.370/src/object.c:702
#11 0x00000c47fcbe2ac9 in debug_master () at
/home/schoelle/Code/Pike-v8.0.370/src/object.c:720
#12 0x00000c47fcbcdc2a in load_pike_master ()
     at /home/schoelle/Code/Pike-v8.0.370/src/pike_embed.c:397
#13 0x00000c47fcb2b794 in main (argc=6, argv=0x7f7ffffd7618)
     at /home/schoelle/Code/Pike-v8.0.370/src/main.c:676

Anybody any hints where I should continue to investigate?

Thanks,
Bernd





Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pike on OpenBSD

Henrik Grubbström-2
On Fri, 30 Dec 2016, Bernd Schoeller wrote:

> Hi -

Hi Bernd and happy new year!

> I would love to run Roxen on my OpenBSD server, and the first step is to get
> a current version of Pike to work on OpenBSD.
[...]

> The first hurdle is compilation of efuns.c:
>
> Compiling modules/_Stdio/efuns.c
> /home/schoelle/Code/Pike-v8.0.370/src/modules/_Stdio/efuns.c: In function
> 'f_get_dir':
> /home/schoelle/Code/Pike-v8.0.370/src/modules/_Stdio/efuns.c:1402: error:
> 'name_max' undeclared (first use in this function)
>
> The corresponding code is:
>
> void f_get_dir(INT32 args)
> {
> [...]
> #ifdef HAVE_READDIR_R
>  ptrdiff_t name_max = -1;
> #endif
>
> [...]
>
>   low_get_dir(dir, name_max);
>   stack_pop_n_elems_keep_top(args);
> }
> #endif /* __NT__ */
>
> So name_max is defined conditionally, but it is used unconditionally.
>
> Removing the #ifdef, compilation continues, but then I get the next error:
Correct fix. Seems I fixed it in Pike 8.1 ~ one and a half years ago,
but forgot to backport it to Pike 8.0.

Fixed.

> Compiling post_modules/GL/top.c
> /home/schoelle/Code/Pike-v8.0.370/build/openbsd-6.0-amd64/tpike
> -DNOT_INSTALLED -DPRECOMPILED_SEARCH_MORE
> -m/home/schoelle/Code/Pike-v8.0.370/build/openbsd-6.0-amd64/master.pike
> /home/schoelle/Code/Pike-v8.0.370/src/post_modules/GL/gen.pike
> /home/schoelle/Code/Pike-v8.0.370/src/post_modules/GL/auto.c.in > auto.c.tmp
> tpike(85013) in free(): error: bogus pointer (double free?) 0xebdd6208440
> Abort trap (core dumped)
[...]
> Anybody any hints where I should continue to investigate?

Hmm... Why is it using tpike in post-modules? You ought to have a full
pike binary at that time. I suspect that you've got several failures
some of which are with the full pike binary.

Anyway, the problematic pointer looks like it could have been valid at
some point in time.

It looks like the crash is during initialization of the master object.

Please try running

   ./pike -t2

and provide the last ~20 lines of the trace.

Thanks,

--
Henrik Grubbström [hidden email]
Roxen Internet Software AB
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pike on OpenBSD

Bernte
Thank you for the feedback.

On 02/01/17 10:38, Henrik Grubbström wrote:
> Correct fix. Seems I fixed it in Pike 8.1 ~ one and a half years ago,
> but forgot to backport it to Pike 8.0.
>
> Fixed.
Great, thanks for the confirmation.
>
> Hmm... Why is it using tpike in post-modules? You ought to have a full
> pike binary at that time. I suspect that you've got several failures
> some of which are with the full pike binary.

The output of running gmake here:

https://www.fams.de/bernte/gmake-out.txt

It currently looks all sound to me: making modules, then compiling and
linking tpike, then making post_modules. I do not see a try to build
'pike'. I don't see any failures, just warning by the compiler.

>
> Anyway, the problematic pointer looks like it could have been valid at
> some point in time.
>
> It looks like the crash is during initialization of the master object.
>
> Please try running
>
>   ./pike -t2
>
> and provide the last ~20 lines of the trace.
As I don't have a running 'pike', I assume this can be done on 'tpike'
as well (my understanding is that tpike is a bootstrap only pike
interpreter with reduced functionality. Is that correct?).

Bernd

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pike on OpenBSD

Chris Angelico
On Tue, Jan 3, 2017 at 9:18 AM, Bernd Schoeller <[hidden email]> wrote:
> The output of running gmake here:
>
> https://www.fams.de/bernte/gmake-out.txt
>
> It currently looks all sound to me: making modules, then compiling and
> linking tpike, then making post_modules. I do not see a try to build 'pike'.
> I don't see any failures, just warning by the compiler.

Does your make build in parallel by default? If there's an odd
dependency problem somewhere, it could be revealed by an out-of-order
build. Try adding -j1 to the make command and see if it's any
different.

ChrisA

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pike on OpenBSD

Henrik Grubbström-2
In reply to this post by Bernte
On Mon, 2 Jan 2017, Bernd Schoeller wrote:

> Thank you for the feedback.
>
> On 02/01/17 10:38, Henrik Grubbström wrote:
>>  Correct fix. Seems I fixed it in Pike 8.1 ~ one and a half years ago,
>>  but forgot to backport it to Pike 8.0.
>>
>>  Fixed.
> Great, thanks for the confirmation.
>>
>>  Hmm... Why is it using tpike in post-modules? You ought to have a full
>>  pike binary at that time. I suspect that you've got several failures
>>  some of which are with the full pike binary.
>
> The output of running gmake here:
>
> https://www.fams.de/bernte/gmake-out.txt
>
> It currently looks all sound to me: making modules, then compiling and
> linking tpike, then making post_modules. I do not see a try to build 'pike'.
> I don't see any failures, just warning by the compiler.
Ah, the configure output explains it;

> checking SO... so
> checking LDSHARED... gcc -Wl,-Bshareable
> checking CCSHARED... checking -fPIC... yes
>  -fPIC
> checking LINKFORSHARED... -Wl,-E
> checking -static-libgcc... yes
> checking if dynamic loading works... no
                                        ^^

So for some reason dlopen() et al don't work as intended, and thus
you get a pike with only static modules, which means that the main
pike binary won't get compiled until after post_modules.

/home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/config.log
should give a hint about why dlopen() failed.

>>  Anyway, the problematic pointer looks like it could have been valid at
>>  some point in time.
>>
>>  It looks like the crash is during initialization of the master object.
>>
>>  Please try running
>>
>>    ./pike -t2
>>
>>  and provide the last ~20 lines of the trace.
> As I don't have a running 'pike', I assume this can be done on 'tpike' as
> well (my understanding is that tpike is a bootstrap only pike interpreter
> with reduced functionality. Is that correct?).
Correct.

--
Henrik Grubbström [hidden email]
Roxen Internet Software AB
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pike on OpenBSD

Bernte
On 03/01/17 09:14, Henrik Grubbström wrote:

> Ah, the configure output explains it;
>
>> checking SO... so
>> checking LDSHARED... gcc -Wl,-Bshareable
>> checking CCSHARED... checking -fPIC... yes
>>  -fPIC
>> checking LINKFORSHARED... -Wl,-E
>> checking -static-libgcc... yes
>> checking if dynamic loading works... no
>                                        ^^
>
> So for some reason dlopen() et al don't work as intended, and thus
> you get a pike with only static modules, which means that the main
> pike binary won't get compiled until after post_modules.
>
> /home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/config.log
> should give a hint about why dlopen() failed.

Ok - that explains it:

configure:68713: checking for dlopen in -ldl
configure:68738:
/home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/smartlink gcc
-o conftest -g  -fvisibility=hidden -O3 -pipe -ggdb -funswitch-loops
-I/usr/local/include -I/usr/X11R6/include
-I/home/schoelle/Code/Pike-v8.0.388/src
-I/home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64
-L/usr/local/lib -R/usr/local/lib -L/usr/X11R6/lib -R/usr/X11R6/lib
-L/usr/lib/gcc-lib/amd64-unknown-openbsd6.0/4.2.1 conftest.c -ldl  -lm  >&5
/usr/bin/ld: cannot find -ldl
collect2: ld returned 1 exit status

Looks like -ldl is not needed on OpenBSD. The functions are available as
documented here:

http://man.openbsd.org/OpenBSD-current/man3/dlfcn.3

I have removed the inclusion of -ldl into LIBS in the configure script.

Now, things are very messy with lots of errors. Will continue to dig
further.

Current output is here:

https://www.fams.de/bernte/gmake-out2.txt
https://www.fams.de/bernte/config.log.txt

Thanks,
Bernd

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Pike on OpenBSD

Henrik Grubbström-2
On Tue, 3 Jan 2017, Bernd Schoeller wrote:

> On 03/01/17 09:14, Henrik Grubbström wrote:
>>  Ah, the configure output explains it;
>>
>> >  checking SO... so
>> >  checking LDSHARED... gcc -Wl,-Bshareable
>> >  checking CCSHARED... checking -fPIC... yes
>> >  -fPIC
>> >  checking LINKFORSHARED... -Wl,-E
>> >  checking -static-libgcc... yes
>> >  checking if dynamic loading works... no
>>                                         ^^
>>
>>  So for some reason dlopen() et al don't work as intended, and thus
>>  you get a pike with only static modules, which means that the main
>>  pike binary won't get compiled until after post_modules.
>>
>>  /home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/config.log
>>  should give a hint about why dlopen() failed.
>
> Ok - that explains it:
>
> configure: 68713: checking for dlopen in -ldl
> configure: 68738:
> /home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/smartlink gcc -o
> conftest -g  -fvisibility=hidden -O3 -pipe -ggdb -funswitch-loops
> -I/usr/local/include -I/usr/X11R6/include
> -I/home/schoelle/Code/Pike-v8.0.388/src
> -I/home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64 -L/usr/local/lib
> -R/usr/local/lib -L/usr/X11R6/lib -R/usr/X11R6/lib
> -L/usr/lib/gcc-lib/amd64-unknown-openbsd6.0/4.2.1 conftest.c -ldl  -lm  >&5
> /usr/bin/ld: cannot find -ldl
> collect2: ld returned 1 exit status
The above is expected and is just a test to see if -ldl is required.

You need to look further down in the config.log.

> Looks like -ldl is not needed on OpenBSD. The functions are available as
> documented here:
>
> http://man.openbsd.org/OpenBSD-current/man3/dlfcn.3
>
> I have removed the inclusion of -ldl into LIBS in the configure script.
>
> Now, things are very messy with lots of errors. Will continue to dig further.
>
> Current output is here:
>
> https: //www.fams.de/bernte/gmake-out2.txt
> https: //www.fams.de/bernte/config.log.txt
From config.log.txt:

> configure:79378: checking if dynamic loading works
> /home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/smartlink gcc -c -g -fvisibility=hidden -O3 -pipe -ggdb -funswitch-loops -pthread -fPIC conftest.c -o conftest.o
> /home/schoelle/Code/Pike-v8.0.388/bin/smartlink gcc -Wl,-Bshareable -L/usr/local/lib -R/usr/local/lib -L/usr/X11R6/lib -R/usr/X11R6/lib -L/usr/lib/gcc-lib/amd64-unknown-openbsd6.0/4.2.1 -pthread conftest.o -o conftest.so
> configure:79430: /home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64/smartlink gcc -o conftest -g  -fvisibility=hidden -O3 -pipe -ggdb -funswitch-loops -pthread -Wl,-E  -I/usr/local/include -I/usr/X11R6/include -I/home/schoelle/Code/Pike-v8.0.388/src -I/home/schoelle/Code/Pike-v8.0.388/build/openbsd-6.0-amd64  -L/usr/local/lib -R/usr/local/lib -L/usr/X11R6/lib -R/usr/X11R6/lib -L/usr/lib/gcc-lib/amd64-unknown-openbsd6.0/4.2.1 -pthread conftest.c -lm  >&5
> In file included from conftest.c:258:
> /home/schoelle/Code/Pike-v8.0.388/src/dynamic_load.c: In function 'main':
> /home/schoelle/Code/Pike-v8.0.388/src/dynamic_load.c:719: warning: incompatible implicit declaration of built-in function 'exit'
> /home/schoelle/Code/Pike-v8.0.388/src/dynamic_load.c:725: warning: incompatible implicit declaration of built-in function 'exit'
> /home/schoelle/Code/Pike-v8.0.388/src/dynamic_load.c:732: warning: incompatible implicit declaration of built-in function 'exit'
> /home/schoelle/Code/Pike-v8.0.388/src/dynamic_load.c:737: warning: incompatible implicit declaration of built-in function 'exit'
> conftest.c: In function 'testfunc2':
> conftest.c:262: warning: incompatible implicit declaration of built-in function 'exit'
> configure:79430: $? = 0
> configure:79430: ./conftest
> conftest:./myconftest.so: undefined symbol 'main'
> Failed to link myconftest.so: Cannot load specified object
> configure:79430: $? = 1
> configure: program exited with status 1
The actual problem is the "./myconftest.so: undefined symbol 'main'",
which indicates that CCSHARED and/or LINKFORSHARED don't have the
correct contents. I suspect that the problem is with CCSHARED, as
AFAIK myconftest.so shouldn't reference main at all.

This looks suspiciously like something I've already fixed in Pike 8.1:

|   if test "$GCC:$ldshared_is_cc" = "yes:yes" ; then
|     # On some architectures gcc attempts to link the shared object with
|     # the standard startup files (crt0.o et al), which may fail due to
|     # relocation errors. The flag -nostartfiles exists in gcc since at
|     # least version 3.3.2 from 2003, so this should be safe.
|     LDSHARED="$LDSHARED -nostartfiles"
|   fi

BTW: I believe that it's probably better to first attempt to get Pike 8.1
      running, and then backport the fixes to Pike 8.0, than the opposite.

      You can always get a fresh Pike 8.1 dist from Pikefarm:
      http://pike.lysator.liu.se/generated/pikefarm/packages/devel/latest

  /grubba

--
Henrik Grubbström [hidden email]
Roxen Internet Software AB
Loading...