Trick to compiling?

Am I doing this wrong?  Do I need clang?  I get tons of u_int vs uint errors, time.h is not sys/time.h - I've got to be doing something terrible wrong.  Tips appreciated, thanks.

On Wednesday, March 2, 2022, helpdeskdan @.***> wrote:

Am I doing this wrong? Do I need clang? I get tons of u_int vs uint
errors, time.h is not sys/time.h - I've got to be doing something terrible
wrong. Tips appreciated, thanks.

yes, i only tried to compile this in clang/termux environment.
can you look around (in Makefile may be) if XOPEN_SOURCE defined somewhere?
if yes - try to remove it?
https://oleherman.com/dev/unknown_type_name/

—
Reply to this email directly, view it on GitHub
https://github.com/Randrianasulu/newfs_udf-linux/issues/1, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AJSS7TJ4ATOQ5UCHMUJSXRLU56CQBANCNFSM5PXZJICQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

The makefile's just a gcc command, there's no XOPEN anywhere in any file.  There's no XOPEN anywhere.  I cobbled the time.h and endian.h, but it's still a bad of worms.  I'm afraid I'm no C programmer.
In file included from udf_create.h:39,
                 from udf_write.c:52:
udf_bswap.h:33:10: fatal error: sys/endian.h: No such file or directory
   33 | #include <sys/endian.h>

On Wednesday, March 2, 2022, helpdeskdan @.***> wrote:

The makefile's just a gcc command, there's no XOPEN anywhere in any file.
There's no XOPEN anywhere. I cobbled the time.h and endian.h, but it's
still a bad of worms. I'm afraid I'm no C programmer.
In file included from udf_create.h:39,
from udf_write.c:52:
udf_bswap.h:33:10: fatal error: sys/endian.h: No such file or directory
33 | #include <sys/endian.h>

try to  add few more places with inclydes (gcc -I..)?
https://github.com/protocolbuffers/protobuf/issues/8373
might give some info about where those files located on different
systems...

—
Reply to this email directly, view it on GitHub
https://github.com/Randrianasulu/newfs_udf-linux/issues/1#issuecomment-1057200811,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJSS7TOHTLQD4HCRAVHWECTU56R6FANCNFSM5PXZJICQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Like I said, I got rid of a couple errors, but there's a lot.  Got time to look?
gcc *.c -o newfs_udf
newfs_udf.c: In function ‘usage’:
newfs_udf.c:824:57: warning: implicit declaration of function ‘getprogname’ [-Wimplicit-function-declaration]
  824 |      "[-t gmtoff] [-v min_udf] [-V max_udf] special\n", getprogname());
      |                                                         ^~~~~~~~~~~
newfs_udf.c:822:33: warning: format ‘%s’ expects argument of type ‘char *’, but argument 3 has type ‘int’ [-Wformat=]
  822 |  (void)fprintf(stderr, "Usage: %s [-cFM] [-L loglabel] "
      |                                ~^
      |                                 |
      |                                 char *
      |                                %d
  823 |      "[-P discid] [-S sectorsize] [-s size] [-p perc] "
  824 |      "[-t gmtoff] [-v min_udf] [-V max_udf] special\n", getprogname());
      |                                                         ~~~~~~~~~~~~~
      |                                                         |
      |                                                         int
newfs_udf.c: In function ‘main’:
newfs_udf.c:840:2: warning: implicit declaration of function ‘setprogname’ [-Wimplicit-function-declaration]
  840 |  setprogname(argv[0]);
      |  ^~~~~~~~~~~
newfs_udf.c:852:26: warning: implicit declaration of function ‘time’; did you mean ‘utimes’? [-Wimplicit-function-declaration]
  852 |  srandom((unsigned long) time(NULL));
      |                          ^~~~
      |                          utimes
newfs_udf.c:865:7: warning: implicit declaration of function ‘localtime’ [-Wimplicit-function-declaration]
  865 |  tm = localtime(&now);
      |       ^~~~~~~~~
newfs_udf.c:865:5: warning: assignment to ‘struct tm *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  865 |  tm = localtime(&now);
      |     ^
newfs_udf.c:866:21: error: dereferencing pointer to incomplete type ‘struct tm’
  866 |  context.gmtoff = tm->tm_gmtoff;
      |                     ^~
udf_create.c:33:9: error: expected declaration specifiers or ‘...’ before string constant
   33 | __RCSID("$NetBSD: udf_create.c,v 1.30 2021/08/20 20:25:26 andvar Exp $");
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from udf_create.c:46:
unicode.h: In function ‘wget_utf8’:
unicode.h:69:53: error: expected declaration specifiers before ‘__unused’
   69 | static u_int16_t wget_utf8(const char **, size_t *) __unused;
      |                                                     ^~~~~~~~
unicode.h:70:49: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__unused’
   70 | static int wput_utf8(char *, size_t, u_int16_t) __unused;
      |                                                 ^~~~~~~~
unicode.h:78:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
   78 | {
      | ^
unicode.h:132:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
  132 | {
      | ^
In file included from udf_create.h:33,
                 from udf_create.c:47:

On Wednesday, March 2, 2022, helpdeskdan @.***> wrote:

Like I said, I got rid of a couple errors, but there's a lot. Got time to
look?
gcc *.c -o newfs_udf
newfs_udf.c: In function ‘usage’:
newfs_udf.c:824:57: warning: implicit declaration of function ‘getprogname’ [-Wimplicit-function-declaration]
824 |      "[-t gmtoff] [-v min_udf] [-V max_udf] special\n", getprogname());
  |                                                         ^~~~~~~~~~~

newfs_udf.c:822:33: warning: format ‘%s’ expects argument of type ‘char *’, but argument 3 has type ‘int’ [-Wformat=]
822 |  (void)fprintf(stderr, "Usage: %s [-cFM] [-L loglabel] "
  |                                ~^

  |                                 |

  |                                 char *

  |                                %d

823 |      "[-P discid] [-S sectorsize] [-s size] [-p perc] "
824 |      "[-t gmtoff] [-v min_udf] [-V max_udf] special\n", getprogname());
  |                                                         ~~~~~~~~~~~~~

  |                                                         |

  |                                                         int

newfs_udf.c: In function ‘main’:
newfs_udf.c:840:2: warning: implicit declaration of function ‘setprogname’ [-Wimplicit-function-declaration]
840 |  setprogname(argv[0]);
  |  ^~~~~~~~~~~

newfs_udf.c:852:26: warning: implicit declaration of function ‘time’; did you mean ‘utimes’? [-Wimplicit-function-declaration]
852 |  srandom((unsigned long) time(NULL));
  |                          ^~~~

  |                          utimes

newfs_udf.c:865:7: warning: implicit declaration of function ‘localtime’ [-Wimplicit-function-declaration]
865 |  tm = localtime(&now);
  |       ^~~~~~~~~

newfs_udf.c:865:5: warning: assignment to ‘struct tm *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
865 |  tm = localtime(&now);
  |     ^

newfs_udf.c:866:21: error: dereferencing pointer to incomplete type ‘struct tm’

yeah, time.h probably differs... between termux and real Linux
will look into this

866 |  context.gmtoff = tm->tm_gmtoff;
  |                     ^~

udf_create.c:33:9: error: expected declaration specifiers or ‘...’ before string constant
33 | __RCSID("$NetBSD: udf_create.c,v 1.30 2021/08/20 20:25:26 andvar Exp $");
  |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In file included from udf_create.c:46:
unicode.h: In function ‘wget_utf8’:
unicode.h:69:53: error: expected declaration specifiers before ‘__unused’
69 | static u_int16_t wget_utf8(const char **, size_t *) __unused;
  |                                                     ^~~~~~~~

unicode.h:70:49: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before ‘__unused’

try to add
#define __unused attribute((unused))
before those lines ?
idea copied from bionic libc/termux's sys/cdefs.h

70 | static int wput_utf8(char *, size_t, u_int16_t) __unused;
  |                                                 ^~~~~~~~

unicode.h:78:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before ‘{’ token
78 | {
  | ^

unicode.h:132:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before ‘{’ token
132 | {
  | ^

In file included from udf_create.h:33,
             from udf_create.c:47:

—
Reply to this email directly, view it on GitHub
https://github.com/Randrianasulu/newfs_udf-linux/issues/1#issuecomment-1057273821,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJSS7TLGHLE3D7TDRB2SNO3U5627RANCNFSM5PXZJICQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

there is also libbsd
https://libbsd.freedesktop.org/wiki/
via
https://stackoverflow.com/questions/12439358/setprogname-in-linux

gcc *.c -o newfs_udf `pkg-config --cflags libbsd-overlay
Well... that does seem to help a little.  The unused def helps.  Let's see.... put a "stdint.h" here, a sys/time.h there.... linking error!  Nuts!  getprogname is in bsd/stdlib.h, not sure how to get it work.
gcc *.c -o newfs_udf `pkg-config --cflags libbsd-overlay`
udf_create.c: In function ‘udf_calculate_disc_layout’:
udf_create.c:412:48: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  412 |  printf("Total space on this medium approx. %llu KiB, %llu MiB\n",
      |                                             ~~~^
      |                                                |
      |                                                long long unsigned int
      |                                             %lu
  413 |    kbsize/1024, kbsize/(1024*1024));
      |    ~~~~~~~~~~~
      |          |
      |          uint64_t {aka long unsigned int}
udf_create.c:412:58: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  412 |  printf("Total space on this medium approx. %llu KiB, %llu MiB\n",
      |                                                       ~~~^
      |                                                          |
      |                                                          long long unsigned int
      |                                                       %lu
  413 |    kbsize/1024, kbsize/(1024*1024));
      |                 ~~~~~~~~~~~~~~~~~~
      |                       |
      |                       uint64_t {aka long unsigned int}
udf_create.c:416:47: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  416 |  printf("Free space on this volume approx. %llu KiB, %llu MiB\n\n",
      |                                            ~~~^
      |                                               |
      |                                               long long unsigned int
      |                                            %lu
  417 |    kbsize/1024, kbsize/(1024*1024));
      |    ~~~~~~~~~~~
      |          |
      |          uint64_t {aka long unsigned int}
udf_create.c:416:57: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  416 |  printf("Free space on this volume approx. %llu KiB, %llu MiB\n\n",
      |                                                      ~~~^
      |                                                         |
      |                                                         long long unsigned int
      |                                                      %lu
  417 |    kbsize/1024, kbsize/(1024*1024));
      |                 ~~~~~~~~~~~~~~~~~~
      |                       |
      |                       uint64_t {aka long unsigned int}
udf_create.c: In function ‘udf_add_logvol_part_sparable’:
udf_create.c:980:11: warning: taking address of packed member of ‘struct part_map_spare’ may result in an unaligned pointer value [-Waddress-of-packed-member]
  980 |  st_pos = &pmap->pms.st_loc[0];
      |           ^~~~~~~~~~~~~~~~~~~~
udf_create.c: In function ‘udf_update_lvintd’:
udf_create.c:1203:8: warning: taking address of packed member of ‘struct logvol_int_desc’ may result in an unaligned pointer value [-Waddress-of-packed-member]
 1203 |  pos = &lvid->tables[0];
      |        ^~~~~~~~~~~~~~~~
udf_create.c: In function ‘udf_extattr_append_internal’:
udf_create.c:1784:23: warning: taking address of packed member of ‘struct impl_extattr_entry’ may result in an unaligned pointer value [-Waddress-of-packed-member]
 1784 |   spos = (uint16_t *) implext->data;
      |                       ^~~~~~~
/usr/bin/ld: /tmp/ccudWxD2.o: in function `usage':
newfs_udf.c:(.text+0x121d): undefined reference to `getprogname'
/usr/bin/ld: /tmp/ccudWxD2.o: in function `main':
newfs_udf.c:(.text+0x1282): undefined reference to `setprogname'
/usr/bin/ld: newfs_udf.c:(.text+0x16ec): undefined reference to `bsd_getopt'
collect2: error: ld returned 1 exit status
make: *** [Makefile:2: all] Error 1

On Thursday, March 3, 2022, helpdeskdan @.***> wrote:

gcc *.c -o newfs_udf `pkg-config --cflags libbsd-overlay
Well... that does seem to help a little. The unused def helps. Let's
see.... put a "stdint.h" here, a sys/time.h there.... linking error! Nuts!
getprogname is in bsd/stdlib.h, not sure how to get it work.

add also 'pkg-config --libs libbsd-overlay'?

gcc *.c -o newfs_udf pkg-config --cflags libbsd-overlay
udf_create.c: In function ‘udf_calculate_disc_layout’:
udf_create.c:412:48: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
412 |  printf("Total space on this medium approx. %llu KiB, %llu MiB\n",
  |                                             ~~~^

  |                                                |

  |                                                long long unsigned int

  |                                             %lu

413 |    kbsize/1024, kbsize/(1024*1024));
  |    ~~~~~~~~~~~

  |          |

  |          uint64_t {aka long unsigned int}

udf_create.c:412:58: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
412 |  printf("Total space on this medium approx. %llu KiB, %llu MiB\n",
  |                                                       ~~~^

  |                                                          |

  |                                                          long long unsigned int

  |                                                       %lu

413 |    kbsize/1024, kbsize/(1024*1024));
  |                 ~~~~~~~~~~~~~~~~~~

  |                       |

  |                       uint64_t {aka long unsigned int}

udf_create.c:416:47: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
416 |  printf("Free space on this volume approx. %llu KiB, %llu MiB\n\n",
  |                                            ~~~^

  |                                               |

  |                                               long long unsigned int

  |                                            %lu

417 |    kbsize/1024, kbsize/(1024*1024));
  |    ~~~~~~~~~~~

  |          |

  |          uint64_t {aka long unsigned int}

udf_create.c:416:57: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
416 |  printf("Free space on this volume approx. %llu KiB, %llu MiB\n\n",
  |                                                      ~~~^

  |                                                         |

  |                                                         long long unsigned int

  |                                                      %lu

417 |    kbsize/1024, kbsize/(1024*1024));
  |                 ~~~~~~~~~~~~~~~~~~

  |                       |

  |                       uint64_t {aka long unsigned int}

udf_create.c: In function ‘udf_add_logvol_part_sparable’:
udf_create.c:980:11: warning: taking address of packed member of ‘struct part_map_spare’ may result in an unaligned pointer value [-Waddress-of-packed-member]
980 |  st_pos = &pmap->pms.st_loc[0];
  |           ^~~~~~~~~~~~~~~~~~~~

udf_create.c: In function ‘udf_update_lvintd’:
udf_create.c:1203:8: warning: taking address of packed member of ‘struct logvol_int_desc’ may result in an unaligned pointer value [-Waddress-of-packed-member]
1203 |  pos = &lvid->tables[0];
  |        ^~~~~~~~~~~~~~~~

udf_create.c: In function ‘udf_extattr_append_internal’:
udf_create.c:1784:23: warning: taking address of packed member of ‘struct impl_extattr_entry’ may result in an unaligned pointer value [-Waddress-of-packed-member]
1784 |   spos = (uint16_t *) implext->data;
  |                       ^~~~~~~

/usr/bin/ld: /tmp/ccudWxD2.o: in function `usage':
newfs_udf.c:(.text+0x121d): undefined reference to `getprogname'
/usr/bin/ld: /tmp/ccudWxD2.o: in function `main':
newfs_udf.c:(.text+0x1282): undefined reference to `setprogname'
/usr/bin/ld: newfs_udf.c:(.text+0x16ec): undefined reference to `bsd_getopt'
collect2: error: ld returned 1 exit status
make: *** [Makefile:2: all] Error 1
—
Reply to this email directly, view it on GitHub
https://github.com/Randrianasulu/newfs_udf-linux/issues/1#issuecomment-1058149583,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJSS7TJIMY5ULLW5RYY7CN3U6DKG7ANCNFSM5PXZJICQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Say, that might have done it!
Usage: newfs_udf [-cFM] [-L loglabel] [-P discid] [-S sectorsize] [-s size] [-p perc] [-t gmtoff] [-v min_udf] [-V max_udf] special

Few warnings:
gcc *.c -o newfs_udf `pkg-config --cflags libbsd-overlay pkg-config --libs libbsd-overlay`
udf_create.c: In function ‘udf_calculate_disc_layout’:
udf_create.c:412:48: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  412 |  printf("Total space on this medium approx. %llu KiB, %llu MiB\n",
      |                                             ~~~^
      |                                                |
      |                                                long long unsigned int
      |                                             %lu
  413 |    kbsize/1024, kbsize/(1024*1024));
      |    ~~~~~~~~~~~
      |          |
      |          uint64_t {aka long unsigned int}
udf_create.c:412:58: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  412 |  printf("Total space on this medium approx. %llu KiB, %llu MiB\n",
      |                                                       ~~~^
      |                                                          |
      |                                                          long long unsigned int
      |                                                       %lu
  413 |    kbsize/1024, kbsize/(1024*1024));
      |                 ~~~~~~~~~~~~~~~~~~
      |                       |
      |                       uint64_t {aka long unsigned int}
udf_create.c:416:47: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  416 |  printf("Free space on this volume approx. %llu KiB, %llu MiB\n\n",
      |                                            ~~~^
      |                                               |
      |                                               long long unsigned int
      |                                            %lu
  417 |    kbsize/1024, kbsize/(1024*1024));
      |    ~~~~~~~~~~~
      |          |
      |          uint64_t {aka long unsigned int}
udf_create.c:416:57: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t’ {aka ‘long unsigned int’} [-Wformat=]
  416 |  printf("Free space on this volume approx. %llu KiB, %llu MiB\n\n",
      |                                                      ~~~^
      |                                                         |
      |                                                         long long unsigned int
      |                                                      %lu
  417 |    kbsize/1024, kbsize/(1024*1024));
      |                 ~~~~~~~~~~~~~~~~~~
      |                       |
      |                       uint64_t {aka long unsigned int}
udf_create.c: In function ‘udf_add_logvol_part_sparable’:
udf_create.c:980:11: warning: taking address of packed member of ‘struct part_map_spare’ may result in an unaligned pointer value [-Waddress-of-packed-member]
  980 |  st_pos = &pmap->pms.st_loc[0];
      |           ^~~~~~~~~~~~~~~~~~~~
udf_create.c: In function ‘udf_update_lvintd’:
udf_create.c:1203:8: warning: taking address of packed member of ‘struct logvol_int_desc’ may result in an unaligned pointer value [-Waddress-of-packed-member]
 1203 |  pos = &lvid->tables[0];
      |        ^~~~~~~~~~~~~~~~
udf_create.c: In function ‘udf_extattr_append_internal’:
udf_create.c:1784:23: warning: taking address of packed member of ‘struct impl_extattr_entry’ may result in an unaligned pointer value [-Waddress-of-packed-member]
 1784 |   spos = (uint16_t *) implext->data;
      |                       ^~~~~~~

Granted, Linux lacking UDF 2.5 write support, I've accomplished rather a Pyrrhic victory with no ability to put anything on said filesystem....

On Thursday, March 3, 2022, helpdeskdan @.***> wrote:

Granted, Linux lacking UDF 2.5 write support, I've accomplished rather a
Pyrrhic victory with no ability to put anything on said filesystem....

and sadly makefs can't put anything in 2.01+  images too...
I used this line for booting netbsd livecd and then mounting device from
inside VM (i used VNC because it was faster on termux, you can use usual
sdl/gtk3 interface..)
qemu-system-i386 -device sb16 -nic user,model=ne2k_pci
-accel tcg,tb-size=64 -vnc :1 -boot d -vga cirrus
-cpu pentium2 -m 320 -hda udfimg.img -cdrom netbsd-92-boot.iso
but netbsd's drive notation a bit different from Linux, you better to
consult netbsd's man online
https://man.netbsd.org/NetBSD-5.0/mount_udf.8
in our case it will be wd0* device (not sure wich one exactly)

—
Reply to this email directly, view it on GitHub
https://github.com/Randrianasulu/newfs_udf-linux/issues/1#issuecomment-1058240809,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJSS7TKCXYSCI7AAAWBEND3U6DTC7ANCNFSM5PXZJICQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

2.01 is supposed to be R/W in Linux.  (Disclaimer: Haven't tried)


eslint the whole source dir

Just a quck fix to make "npm run lint" lint the whole src directory + don't make npm fail with a failure code

Thanks for the PR again 😁


Fix imprecise statements for virtual hosting example.

In the provided ingress example, the traffic with other host value will be forwarded to service3.
But the original doc is that only the traffic without host header can be forwarded to service3, which is not correct.
Moreover, the host header is required for request. When the target is ip address, the host value would be ip address. Otherwise, it should be a valid domain.

/check-cla

Hi. We recommend restricting fixes to a single localization. Would you be willing to revise this so it's just the English changes, @SpecialYang?

/approve
Thanks @SpecialYang


reply button isn't working.

Clicking on the reply to button end in timeout.

Mhm after some more clicks, it is working....strange.


[master] RO-3976 Remove mariadb.yml from group_vars

MariaDB was fixed upstream so we neeed to delete the variable file.
Issue: RO-3976

are we consuming the fix though (didn't think we had time to update our sha).


ImportError: no module named mac

After cloning the directory, entering it, and typing 'pip install .' I got what seemed to be a successful build and install, ending in:
Successfully built Send2Trash
Installing collected packages: Send2Trash
Successfully installed Send2Trash-1.8.1b0

But when I try to run my Python 2.7 script I get an error message:
Traceback (most recent call last):
  File "/Users/larryy/Dropbox/src/file_rename.py", line 8, in <module>
    from send2trash import send2trash
  File "/usr/local/lib/python2.7/site-packages/send2trash/__init__.py", line 12, in <module>
    from .mac import send2trash
ImportError: No module named mac

As shown, the line of code in question is a standard import:
from send2trash import send2trash

This is macOS Monterey 12.4 (latest) and the latest Send2Trash (1.8.1b0).

Likely an issue related to https://github.com/arsenetar/send2trash/commit/d37197c4f7e3e7fa083930aaac0841d1895db134 although there should not be issues... I recommend trying the 1.8.1b0 tag instead of the master branch. to see if that resolves the issue.

This should be good now, I needed to update the setup.cfg.

I'm sorry to report that "fix" doesn't help.  I installed the latest stable releases for both Python 3 (3.9.13) and Python 2 (2.7.18), reinstalled send2trash, and am getting the same error.  The only customizable options were certain packages, which were all on by default and I left them on, and the install location, which I left at the default.  I confirmed that Python 2 was updated from 2.7.17 to 2.7.18.  I don't know what version Python 3 was at before this install, but it is now at 3.9.13.
I don't think it's meaningful, but I noticed a complaint from the send2trash install about the target site-packages not being in my PATH, and sure enough I still had an old /System/Library path instead of the current /Library path.  But fixing this seems to have changed nothing except that paths reported in the error message are now the Framework paths rather than the /usr/local/lib paths.  The files being pointed to are the same files, just reachable by these two different paths due to symbolic linking.
Traceback (most recent call last):
File "/Users/larryy/Dropbox/src/file_rename.py", line 8, in
from send2trash import send2trash
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/send2trash/init.py", line 12, in
from send2trash.mac import send2trash
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/send2trash/mac/init.py", line 20, in
from send2trash.mac.legacy import send2trash  # noqa: F401
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/send2trash/mac/legacy.py", line 18, in
GetMacOSStatusCommentString = Foundation.GetMacOSStatusCommentString
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/init.py", line 379, in getattr
func = self.getitem(name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/init.py", line 384, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, GetMacOSStatusCommentString): symbol not found

Yeah, I don't really have anything else then, installing python 3.9.2 back when I was first troubleshooting the symbol not found issue somehow "fixed" the system pythons on my development mac which no longer has this issue to date with any of the three python versions present.  I would encourage using a newer version of python if you can.


accordion cleanup. remove the deprecated accordion component.

You did it @dinakarmeda!
Thank you for signing the Anthem Opensource Software CLA.


feat(headless): add answerTextIsEmpty to genQA UA stream event

SVCC-3542
Add answerTextIsEmpty to genQA UA events In order to check whether there was a message or not.
open PR in Coveo.analytics to add this new parameter to the interface here: PR
In the meantine added a Type checker .

🤔 wouldn't it be the right time to apply scout camp principles and ensure the action is available with the new analytic protocol?

The GeneratedAnswerStreamEnd event is one of the events that will be deprecated with the new event protocol, it's actually for customers who don t use the new event protocol.


Reply Icon missing

The icon when a user replies or "@" mentions someone is missing as seen here in this screenshot:
https://imgur.com/a/FdeNIx4

This is related to #121. It will be fixed when that gets merged.

@Nathan2023 Fixed this with 2a61dfa


"No frame received" on intel realsense viewer in PC

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
{ L515 }

Firmware Version
(Open RealSense Viewer  v2.41.0)

Operating System & Version
{Win (10)

Platform
PC

Language
{C/C# }

Issue Description
I have a problem using the L515 camera. When the intel Realsense viewer is turned on the depth, IR, confidence streams in the depth sensor will fail (No frame received!) for 1-2 seconds, then it works normally. But with RGB sensor, it works well

I'd appreciate any help with this. Thanks!

Hi @nh-97
Have a couple of questions based on what you're seeing:

Are you seeing the message only at initial enabling of the streams?  I'm thinking its the initial frames until the device gets up and running.
I notice that the RS Viewer you're using is 2.38.1, are you seeing this same issue on the latest Viewer 2.42?
What FW do you have on your device?  Can you show the "info" in the Viewer?
Also provide the settings in which you have for the depth streams?

Thanks @RealSenseSupport !

Chào @ nh-97
Có một số câu hỏi dựa trên những gì bạn đang thấy:

Bạn có thấy thông báo chỉ khi bật luồng ban đầu không? Tôi đang nghĩ về những khung hình ban đầu cho đến khi thiết bị khởi động và chạy.
When streaming it does not show any message. Yep, don't get the first frames.
Tôi nhận thấy rằng Trình xem RS bạn đang sử dụng là 2.38.1, bạn có gặp sự cố tương tự trên Trình xem 2.42 mới nhất không?
I have updated RS Viewer to v2.40.1, v2.41.0, v2.42.0 but still have this problem. Also update the Firmware accordingly..
Bạn có FW nào trên thiết bị của mình? Bạn có thể hiển thị "thông tin" trong Trình xem không?
This is information about the device

Đồng thời cung cấp các cài đặt mà bạn có cho các luồng chiều sâu?
Are you wanting to know about these settings?

Besides, I have checked the usb caple (replace another usb cable 3.1 and another usb port), connect the camera to another PC. But that doesn't change the problem.

Hi @nh-97
Have a couple of questions based on what you're seeing:

Are you seeing the message only at initial enabling of the streams?  I'm thinking its the initial frames until the device gets up and running.
When streaming it does not show any message. Yep, don't get the first frames.
I notice that the RS Viewer you're using is 2.38.1, are you seeing this same issue on the latest Viewer 2.42?

I have updated RS Viewer to v2.40.1, v2.41.0, v2.42.0 but still have this problem. Also update the Firmware accordingly..

What FW do you have on your device?  Can you show the "info" in the Viewer?
This is information about the device

Also provide the settings in which you have for the depth streams?
Are you wanting to know about these settings?

Besides, I have checked the usb cable (replace another usb cable 3.1 and another usb port), connect the camera to another PC. But that doesn't change the problem.

Hi @nh-97
Have a couple of questions based on what you're seeing:

Are you seeing the message only at initial enabling of the streams?  I'm thinking its the initial frames until the device gets up and running.
When streaming it does not show any message. Yep, don't get the first frames.
I notice that the RS Viewer you're using is 2.38.1, are you seeing this same issue on the latest Viewer 2.42?

I have updated RS Viewer to v2.40.1, v2.41.0, v2.42.0 but still have this problem. Also update the Firmware accordingly..

What FW do you have on your device?  Can you show the "info" in the Viewer?
This is information about the device

Also provide the settings in which you have for the depth streams?
Are you wanting to know about these settings?

Besides, I have checked the usb cable (replace another usb cable 3.1 and another usb port), connect the camera to another PC. But that doesn't change the problem.


Multilingual Translation FR-EN, DE-EN

❓ Questions and Help
Before asking:

search the issues.
search the docs.

What is your question?
Hello, i'm using the  code availabe on https://github.com/pytorch/fairseq/tree/master/examples/translation in order to train a multilingual translator FR-EN, DE-EN i trained it using the dataset available in the code and the same hyper parameters however once loaded, the translations are not good at all. The sentences that i'm trying to translate are very domain specific can this be the source of the problem ?
Code
mkdir -p checkpoints/multilingual_transformer
CUDA_VISIBLE_DEVICES=0 fairseq-train data-bin/iwslt17.de_fr.en.bpe16k/
--max-epoch 50
--ddp-backend=no_c10d
--task multilingual_translation --lang-pairs de-en,fr-en
--arch multilingual_transformer_iwslt_de_en
--share-decoders --share-decoder-input-output-embed
--optimizer adam --adam-betas '(0.9, 0.98)'
--lr 0.0005 --lr-scheduler inverse_sqrt --min-lr '1e-09'
--warmup-updates 4000 --warmup-init-lr '1e-07'
--label-smoothing 0.1 --criterion label_smoothed_cross_entropy
--dropout 0.3 --weight-decay 0.0001
--save-dir checkpoints/multilingual_transformer
--max-tokens 4000
--update-freq 8

What have you tried?
What's your environment?

fairseq Version (e.g., 1.0 or master):
PyTorch Version (e.g., 1.0)
OS (e.g., Linux):
How you installed fairseq (pip, source):
Build command you used (if compiling from source):
Python version:
CUDA/cuDNN version:
GPU models and configuration:
Any other relevant information:

How does the model do when translating sentences in the validation set?  This should give you a better sense of whether or not the model is bad or your sentences are too out of domain.

Thanks for the response !
I'm getting 3.728 loss and 2.119 nll_loss for the validation set

I mean in terms of BLEU or even just manually inspecting the translations.  Do they seem sensible?

i'm getting an error when i try to generate and score the test set with sacrebleu with the code on https://github.com/pytorch/fairseq/tree/master/examples/translation
The error i'm getting is sacreBLEU: Test set iwslt17 contains no sentence


About extract event

hello. it's wonderful works!
in the paper, there is event retrieval by using the img or text query.

can you inform "how to retrieval the event from other modality query"?
thank you.

Please refer to the evaluate function in the train_dp_datasetname.py for code implementation.

thank you for your answering.
But, I want to know "how to visualize the events".
can you share the code for visualize it?

For the visualization of events, you may refer to the data loader part, where codes related to transforming event to event frames are released.


Add friend list feature

People have friends. So implement it.

Each user has a list of friends eg friends = [PersonA, PersonC]


command line to hygrate tw user details

https://developer.twitter.com/en/docs/twitter-api/users/lookup/quick-start/user-lookup
can do automatically

not going to happen


Show expected number of blocks per Shelley epoch

In Shelley, the number of slots and blocks per epoch will never match so the current way we display blocks / epochs doesn't really make sense anymore. What we need instead is to display the expected number of blocks within an epoch (and not use the slots value anymore)

The expected number of blocks is probabilistic, so it's not practical to achieve what we want here. Instead, let's display each value in separate columns to remove the direction of the blocks being a 1:1 target with slots.


Manage GitOps Catalog uninstalling of application removed manually

Which version of kubefirst are you using?
2.3.8
Which cloud provider?
None specific
Which DNS?
None specific
Which installation type?
None specific
Which distributed Git provider?
None specific
Did you use a fork of gitops-template?
No
Which Operating System?
None specific
What is the issue?
If one try to uninstall an application that was deleted directly in the repository by the user, nothing show in the UI, but there is an error in the browser console.
https://github.com/kubefirst/kubefirst/assets/1170989/464c10a9-867c-4583-b210-9979d7c8c372
We should check if it's still there, and show an error to the user. If it's the fact that it was already deleted, maybe even remove it from the console and telling the user it was already deleted or something along the lines.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Here is what @donnalux proposed on Slack:


Pipeline Testing

Double checked already that exome bam and my BED file have compatible contigs but no success unfortunately:
leus/io/sam_reader.cc:537] Unrecognized SAM header type, ignoring:
  I1022 11:53:51.789329<PHONE_NUMBER>2080 genomics_reader.py:213] Reading hroc113_0.md.bam with NativeSamReader
  I1022 11:53:51.808645<PHONE_NUMBER>2080 make_examples.py:991] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
  I1022 11:53:51.819024<PHONE_NUMBER>2080 genomics_reader.py:213] Reading exons_padded50bp_ucsc_hg19_grch37.bed with NativeBedReader
  I1022 11:54:03.196350<PHONE_NUMBER>2080 ranges.py:121] Adding interval chr6:117127483-117128419 to intervaltree
  I1022 11:54:14.932882<PHONE_NUMBER>2080 ranges.py:121] Adding interval chr14:24034765-24034960 to intervaltree
  Traceback (most recent call last):
    File "/scratch/5306779/Bazel.runfiles_7WvCJa/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1188, in <module>
      tf.app.run()
    File "/opt/conda/envs/nf-core-deepvariant-1.0dev/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 125, in run
      _sys.exit(main(argv))
    File "/scratch/5306779/Bazel.runfiles_7WvCJa/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1178, in main
      make_examples_runner(options)
    File "/scratch/5306779/Bazel.runfiles_7WvCJa/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1076, in make_examples_runner
      regions = processing_regions_from_options(options)
    File "/scratch/5306779/Bazel.runfiles_7WvCJa/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 995, in processing_regions_from_options
      raise ValueError('The regions to call is empty. Check your --regions and '
  ValueError: The regions to call is empty. Check your --regions and --exclude_regions flags to make sure they are not resulting in set of empty region to process. This also happens if you use "chr20" for a BAM where contig names don't have "chr"s (or vice versa).
  parallel: This job failed:
  /opt/conda/envs/nf-core-deepvariant-1.0dev/bin/python /opt/conda/envs/nf-core-deepvariant-1.0dev/share/deepvariant-0.7.0-0/binaries/DeepVariant/0.7.0/DeepVariant-0.7.0+cl-208818123/make_examples.zip --mode calling --ref human_g1k_v37_decoy.fasta.gz --reads hroc113_0.md.bam --regions exons_padded50bp_ucsc_hg19_grch37.bed --examples<EMAIL_ADDRESS>--task 0
  Warning: unable to close filehandle properly: Remote I/O error during global destruction.

Hi @apeltzer,
Would it be possible for you to send the exome bam and BED file? Is the bam file small?
Thanks in advance

BED would be possible, but not the bam (private data) - Anything I can do to find out what's wrong?

The problem isn't with the exome model wes because this runs successfully:
nextflow run main.nf --exome -profile test,docker

I know you said you've checked the bam and bed files but I think there must be some kind of problem with the data.
The input bam file for make_examples use the convention without the prefix "chr":
Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']

Does the bed file specify them in the same way?

I'll try once more!
This was the command:
nextflow run deepvariant/main.nf -profile binac,singularity --fasta /beegfs/work/iiipe01/Reference_Genomes/Human/GRCH37-Sarek/human_g1k_v37_decoy.fasta --bam hroc113_0.md.bam --bed /beegfs/work/iiipe01/Reference_Genomes/Human/GRCH37-Sarek/exons_padded50bp_ucsc_hg19_grch37.bed --exome

(had to add our own hardware profile binac here)

Recreated the BED file and will report back...

Probably a corrupt BED issue. Its running now, will report back once the results are there!

Okay thanks and yeah DeepVariant can take a while to run, especially if the BAM file is large

Hey @apeltzer,
I'm assuming the results came back fine. If so do I then have the go-ahead to release deepvariant?

They didn't unfortunately ... it crashed on the same input.
nextflow.log.txt

Hi @apeltzer,
According to your log file the cause was:
Process `make_examples (hroc113_0.md.bam)` terminated for an unknown reason -- Likely it has been terminated by the external system

Is this running on a HPC local cluster?
How big is the bam file/regions being analysed?
I know make_examples is the most computationally intensive process, is it possible that the resources required exceeded the requested resources/limits and so the job was killed?

Hi @apeltzer,
According to your log file the cause was:
Process `make_examples (hroc113_0.md.bam)` terminated for an unknown reason -- Likely it has been terminated by the external system

Is this running on a HPC local cluster?
How big is the bam file/regions being analysed?
I know make_examples is the most computationally intensive process, is it possible that the resources required exceeded the requested resources/limits and so the job was killed?

Hm, could be but the system should have ~28 cores and the input was ~5GB in file size. It has 48hours for running through....

We benchmarked DeepVariant internally at Lifebit before and I think at least 50GB of memory is roughly required for a 5GB BAM file to be sure it runs.
Do you know how much memory you had access to when running DeepVariant?

The machine I use has ~256GB of memory, 28 CPU cores. I configured just a hardware profile for the system, without other changes. Do you specify that you need that much memory in the process configuration?

It's not specified in the process configuration. The amount of memory required is typically about 10-15x the size of the BAM file. I'm not sure exactly how to specify this in the configuration process. I know that task.cpus is used in make_examples. Perhaps, it could be this causing the problem if the value of task.cpus is too low

@apeltzer Okay thanks.
How much memory do you think the job is allocated by default if you did not specify any? Can you also please send the trace file?

As I didn't change it, it should be 8.GB according to your base.config, so I guess that's the biggest issue here.

Please can you try setting the memory in your config file to 50GB and using the with trace option. That way we can test if this is the (only) problem.
I'll try adding this but I'm struggling to get it to work:
file.size() > 1.GB ? {task.memory = 15 * file.size() } : task.memory

There's a few things I'm not clear on this:
Should this belong in the base.config file or the under the relevant process eg make_examples in main.nf?
Presumably I need to retreive the bam from bamChannel before doing this, do I also need to call 'file()' on it?
What if there are multiple bam files?
I'll add this to the documentation and it may be easier to just get users to specify the memory/CPUs etc and give guidelines for doing this

No laptop and I can just have a look at this tomorrow - our Cluster where I ran this is in maintenance till Friday, so can try this then.
I’ll also have a look in detail, traveling atm and mobile phone is not enough to carefully look etc ...

Hi @apeltzer sorry to keep bothering you
Does the code above (for setting the memory) look alright?
If you're are not able to test the pipeline until Friday would it be alright if I see if anyone else from nf-core can test the pipeline before then instead?
Thanks

If there are some other people able to test tha twould be good too - I can't test before Friday as the maintenance goes on for now :-1:

Unfortunately that cluster seems to be down still. Can anyone from @nf-core/core have a go and test this?

Unfortunately that cluster seems to be down still. Can anyone from @nf-core/core have a go and test this?

If anyone does test it, please specify memory in your config file to roughly 10x the size of your BAM file eg 50GB of memory for 5GB BAM file
Thanks

Hi @apeltzer,
Is there any chance the cluster is back up?
Thanks

ssh binac
<EMAIL_ADDRESS>password:
Connection closed by <IP_ADDRESS> port 22
alex@aragorn ~/I/n/e/w/c/c25f32f423ccce54f80c528a174b86>

Sorry no :-1: Getting annoyed by that one too, but can't push them any further to proceed faster ...

Cluster is back, testing today :-)

Running - with 50GB memory for the process, also had to update the cluster modules in a file and will send a PR to make that accessible to everyone.

Okay perfect, thanks

=======================================================
Pipeline Name  : nf-core/deepvariant
Pipeline Version: 1.0dev
Bam file       : hroc113_0.md.bam
Bed file       : /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/WES_targets_centogene.bed
Fasta Ref      : /beegfs/work/iiipe01/Reference_Genomes/Human/GRCH37-Sarek/human_g1k_v37_decoy.fasta
Max Memory     : 128 GB
Max CPUs       : 28
Max Time       : 2d
Model          : wes
Output dir     : results
Working dir    : /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work
Container Engine: singularity
Container      : nfcore/deepvariant:dev
Current home   : /home-link/iiipe01
Current user   : iiipe01
Current path   : /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test
Script dir     : /home-link/iiipe01/.nextflow/assets/nf-core/deepvariant
Config Profile : binac
=========================================
[warm up] executor > pbs
WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work/singularity
Pulling Singularity image docker://nfcore/deepvariant:dev [cache /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work/singularity/nfcore-deepvariant-dev.img]
[51/a21d26] Submitted process > preprocess_fastagz (human_g1k_v37_decoy.fasta.gz)
[ce/0e0d34] Submitted process > preprocess_gzi (human_g1k_v37_decoy.fasta.gz.gzi)
[bc/9e30f8] Submitted process > get_software_versions
[fb/cfb018] Submitted process > preprocess_fai (human_g1k_v37_decoy.fasta.fai)
[90/f3eb1a] Submitted process > preprocess_bam (hroc113_0.md.bam)
[7f/c309b8] Submitted process > preprocess_gzfai (human_g1k_v37_decoy.fasta.gz.fai)

Unfortunately this hangs and there are no jobs submitted anymore...

[90/f3eb1a] Submitted process > preprocess_bam (hroc113_0.md.bam)

That's the bad boy ;-) Pipeline hangs, the other ones ran through properly. Also this is not in the scheduler anymore!

Do you think the errors you mentioned in #3 regarding vim and igenomes.config for the binac profile caused the problem?
Are the max memory and max cpus correct?
Can you also please send the nextflow.log if you have it
Thanks

I will run again right now :-)

Perfect, thanks 😃

I guess we'd have to find something for the file.size() feature, as this causes an error as well:
java.lang.AssertionError: Memory unit cannot be a negative number. Expression: (value >= 0). Values: value = -3789259617654013952
        at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:417)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:672)
        at nextflow.util.MemoryUnit.<init>(MemoryUnit.groovy:54)
        at nextflow.util.MemoryUnit.multiply(MemoryUnit.groovy:118)
        at groovy.runtime.metaclass.NumberDelegatingMetaClass.invokeMethod(NumberDelegatingMetaClass.java:75)
        at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127)
        at Script8691A4FE657AC67848F69F4B9337B93D$_run_closure1$_closure7$_closure11.doCall(Script8691A4FE657AC67848F69F4B9337B93D:26)
        at Script8691A4FE657AC67848F69F4B9337B93D$_run_closure1$_closure7$_closure11.doCall(Script8691A4FE657AC67848F69F4B9337B93D)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:104)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:326)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1041)
:

@apeltzer is this a problem with the file.size() feature itself? As even ignoring the fact that its negative the value is wrong. Do you think the best solution is to change the code back to how it was before and add something in the documentation about setting memory to 10x BAM file size?

No, I don't think that's good - the feature is there and should work :-)
I'll have a look

Hm, I'm a bit wondering what we multiple here:
If the .size() method returns 0, then that won't help. That size is in bytes, so we'd probably have to run .toGiga() to get Gigabytes.
https://www.nextflow.io/docs/latest/script.html?highlight=size#check-file-attributes
  memory = { check_max( bam.size().toGiga() * 5.GB  * task.attempt, 'memory')}

I believe it doesn#t resolve the bam channel properly. Having this by default using some reasonable values for memory would be quite user-friendly instead of forcing people to specify it themselves (without having that experience for example).

Hm, maybe @pditommaso has an idea on how to use this properly. I believe it could also be an issue when we set this in the base.config instead of the main script.

@apeltzer I tried adding toGiga() eg memory = { check_max( bam.size().toGiga() * 5.GB  * task.attempt, 'memory')}
However, when I try running it I get this error:
Exception in thread "Task submitter" groovy.lang.MissingMethodException: No signature of method: java.lang.Long.toGiga() is applicable for argument types: () values: []
Do you think this is a problem with the type of bam.size() which means I can't call toGiga() on it? Do you know how to solve this?
Thanks

Okay no worries, thanks for mentioning it in the gitter chat

I'm hitting another test :-)

I'm getting this here:

Caused by:
  Failed to submit process to grid scheduler for execution

Command executed:

  qsub -N nf-make_example .command.run

Command exit status:
  168

Command output:
  qsub: submit error (Illegal attribute or resource value for Resource_List.mem)

Work dir:
  /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work/d7/78ed3a18fda374310837f6ce93b9e7

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
Execution cancelled -- Finishing pending tasks before exit

Do you know what the requested resources were?

.command.run says:
#PBS -N nf-make_example
#PBS -o /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work/d7/78ed3a18fda374310837f6ce93b9e7/.command.log
#PBS -j oe
#PBS -q short
#PBS -l nodes=1:ppn=20
#PBS -l walltime=10:00:00
#PBS -l mem=57.4gb
cd /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work/d7/78ed3a18fda374310837f6ce93b9e7

(which should be definitely available on that node, having at max 28 cores, 48hours of wall time and 128GB RAM per node...

Do you have any idea what the issue may be?

Not really, I'm pinging @pditommaso in here ... or you may want to open an issue at the main nextflow repository? Never saw that before using the file.size() option

Hi guys, just catching up with notifications. Great thread to read! I mentioned this dynamic feature in passing a while back @PhilPalmer - would be really nice to get it working (and may copy syntax for other pipelines if / when this works). Note that the base.config should have "sensible defaults", and 8.GB probably isn't. So would be good to minimise how many other config files vary these if possible.
Sounds like we're on track though!

@apeltzer I know I've said this a lot but the pipeline should be ready for testing now 😅
If you could try running it again that would be much appreciated, thanks

@ewels are you saying the default 8.GB in base.config is too high? Shall I change it to 4.GB instead?
Eg:
  cpus = { check_max( 1 * task.attempt, 'cpus' ) }
  memory = { check_max( 4.GB * task.attempt, 'memory' ) }
  time = { check_max( 2.h * task.attempt, 'time' ) }

Okay I think I understand. The reason why it worked with the Travis build was that the test BAM file was used which is small and so check_max( bam.size() >> 30 * 10.GB * task.attempt, 'memory') was never executed (see code below).
memory = { bam.size() <<PHONE_NUMBER> ? 4.GB : check_max( bam.size() >> 30 * 10.GB * task.attempt, 'memory')}

And I think it should be check_max( bam.size() >> 5 * 10.GB * task.attempt, 'memory') instead so I'll change that

Perfect thanks 🤞

Looking good :tada:
5369328                    nf-get_software  iiipe01                0 Q short
5369329                    nf-make_example  iiipe01         02:37:44 C short
5372698                    nf-call_variant  iiipe01                0 Q short

(partially hanging in the queue, but thats not your fault :-))

Running well, we're postprocessing now :-)

@apeltzer did it manage to run to completion?

Yes it did!!!
sh runDeepvariant.sh
N E X T F L O W  ~  version 18.10.1
Launching `nf-core/deepvariant` [evil_pesquet] - revision: f24c49cabb [dev]
=======================================================
                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~'
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'
nf-core/deepvariant v1.0dev"
=======================================================
Pipeline Name  : nf-core/deepvariant
Pipeline Version: 1.0dev
Bam file       : hroc113_0.md.bam
Bed file       : /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/WES_targets_centogene.bed
Fasta Ref      : /beegfs/work/iiipe01/Reference_Genomes/Human/GRCH37-Sarek/human_g1k_v37_decoy.fasta
Max Memory     : 128 GB
Max CPUs       : 28
Max Time       : 2d
Model          : wes
Output dir     : results
Working dir    : /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test/work
Container Engine: singularity
Container      : nfcore/deepvariant:latest
Current home   : /home-link/iiipe01
Current user   : iiipe01
Current path   : /beegfs/work/iiipe01/2018-10-21_DeepVariant_Test
Script dir     : /home-link/iiipe01/.nextflow/assets/nf-core/deepvariant
Config Profile : binac
=========================================
[warm up] executor > pbs
[9c/581fc2] Cached process > preprocess_fai (human_g1k_v37_decoy.fasta.fai)
[43/dceebb] Cached process > preprocess_bam (hroc113_0.md.bam)
[6b/3c32b0] Cached process > preprocess_gzi (human_g1k_v37_decoy.fasta.gz.gzi)
[3c/f89af0] Cached process > preprocess_fastagz (human_g1k_v37_decoy.fasta.gz)
[75/e53133] Submitted process > get_software_versions
[ac/f03f86] Cached process > preprocess_gzfai (human_g1k_v37_decoy.fasta.gz.fai)
[74/d5b07b] Submitted process > make_examples (hroc113_0.md.bam)
[a9/e2f586] Submitted process > call_variants (hroc113_0.md.bam)
[2f/599902] Submitted process > postprocess_variants (hroc113_0.md.bam)
a[nf-core/deepvariant] Pipeline Complete! You can find your results in /home-link/iiipe01/.nextflow/assets/nf-core/deepvariant/results
[iiipe01@login03 2018-10-21_DeepVariant_Test]$ a

Which in turns means, I'm happy if you do a release now :-)

Sorry to comment in a closed thread. I'm having the same issue. How is solved, is a memory thing?
$ nextflow run nf-core/deepvariant    --fasta /home/rippell/data/GATK/10consensus.fasta --fai /home/rippell/data/GATK/10consensus.fasta.fai --bam /home/rippell/data/GATK/i7-97_sorted_reads.bam --bed /home/rippell/data/GATK/i7-97_sorted_reads.bed --max_memory 24.GB --max_cpus 3 -profile standard,docker -resume
N E X T F L O W  ~  version 20.04.1
Launching nf-core/deepvariant [infallible_engelbart] - revision: 2b5486356c [master]
executor >  local (1)
[99/9cff25] process > preprocess_fastagz (10consensus.fasta.gz)   [100%] 1 of 1, cached: 1 ✔
[d9/8d8851] process > preprocess_gzfai (10consensus.fasta.gz.fai) [100%] 1 of 1, cached: 1 ✔
[32/8ea296] process > preprocess_gzi (10consensus.fasta.gz.gzi)   [100%] 1 of 1, cached: 1 ✔
[49/356721] process > preprocess_bam (i7-97_sorted_reads.bam)     [100%] 1 of 1, cached: 1 ✔
[f7/d63e97] process > make_examples (i7-97_sorted_reads.bam)      [100%] 1 of 1, failed: 1 ✘
[-        ] process > call_variants                               -
[-        ] process > postprocess_variants                        -
[2a/f81c8e] process > get_software_versions                       [100%] 1 of 1, cached: 1 ✔
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'make_examples (i7-97_sorted_reads.bam)'
Caused by:
Process make_examples (i7-97_sorted_reads.bam) terminated with an error exit status (247)
Command executed:
mkdir logs
mkdir i7-97_sorted_reads_shardedExamples
dv_make_examples.py   --cores 3   --sample i7-97_sorted_reads.bam   --ref 10consensus.fasta.gz   --reads i7-97_sorted_reads.bam   --regions i7-97_sorted_reads.bed   --logdir logs   --examples i7-97_sorted_reads_shardedExamples
Command exit status:
247
Command output:
(empty)
ETA: 0s Left: 3 AVG: 0.00s  local:3/0/100%/0.0s 2020-07-14 02:04:41.102417: W third_party/nucleus/io/sam_reader.cc:537] Unrecognized SAM header type, ignoring:
I0714 02:04:41.361650<PHONE_NUMBER>72832 genomics_reader.py:213] Reading i7-97_sorted_reads.bam with NativeSamReader
I0714 02:04:41.711935<PHONE_NUMBER>72832 make_examples.py:1075] Preparing inputs
2020-07-14 02:04:43.268289: W third_party/nucleus/io/sam_reader.cc:537] Unrecognized SAM header type, ignoring:
I0714 02:04:43.512497<PHONE_NUMBER>72832 genomics_reader.py:213] Reading i7-97_sorted_reads.bam with NativeSamReader
I0714 02:06:47.752322<PHONE_NUMBER>72832 make_examples.py:991] Common contigs are [u'disjointig_1', u'disjointig_10', u'disjointig_100', u'disjointig_1000', u'disjointig_10000', u'disjointig_100000', u'disjointig_100001', u'disjointig_100002', u'disjointig_100003', u'disjointig_100004', u'disjointig_100005', u'disjointig_100006', u'disjointig_100007', u'disjointig_100008', u'disjointig_100009', u'disjointig_10001', u'disjointig_100010', u'disjointig_100011', u'disjointig_100012', u'disjointig_100013', u'disjointig_100014', u'disjointig_100015', u'disjointig_100016', u'disjointig_100017', u'disjointig_100018', u'disjointig_100019', u'disjointig_10002', u'disjointig_100020', u'disjointig_100021', u'disjointig_100022', u'disjointig_100023', u'disjointig_100024', u'disjointig_100025', u'disjointig_100026', u'disjointig_100027', u'disjointig_100028', u'disjointig_100029', u'disjointig_10003', u'disjointig_100030', u'disjointig_100031', u'disjointig_100032', u'disjointig_1000
Command error:
ETA: 0s Left: 3 AVG: 0.00s  local:3/0/100%/0.0s
parallel: This job failed:
/opt/conda/envs/nf-core-deepvariant-1.0/bin/python /opt/conda/envs/nf-core-deepvariant-1.0/share/deepvariant-0.7.0-0/binaries/DeepVariant/0.7.0/DeepVariant-0.7.0+cl-208818123/make_examples.zip --mode calling --ref 10consensus.fasta.gz --reads i7-97_sorted_reads.bam --regions i7-97_sorted_reads.bed --examples<EMAIL_ADDRESS>--task 2
Work dir:
/home/rippell/data/GATK/work/95/514c9ed1d1491443b784e05a064982


Upd aiohttp and codecov

Current coverage is 78.83% (diff: 100%)

No coverage report found for mq-prototype at 78248e3.

Powered by Codecov. Last update 78248e3...5e32cf7


Qemu Device type fails to boot if disk type is VIRTIO or SCSI_VIRTIO

Qemu image has a boot root parameter set to /dev/hda2 causing the boot to fail when the image is run with a disk that is backed by VIRTIO or SCSI_VIRTIO. Changing this to a disk by label path allows for booting in all scenarios.
This PR adds support for the Kernel modules needed for Google Cloud Platform as well. ( SCSI_VIRTIO support )
Closes https://github.com/resin-os/resinos/issues/248

@resin-jenkins retest this please

@agherzan added the meta-reisn submodule update as part of this PR so that we can do the root=LABEL=... switch.
Please review

@resin-jenkins retest this please.


Branch ug

Codecov Report

Merging #211 (7046e82) into master (72b26d5) will decrease coverage by 0.16%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master     #211      +/-   ##
============================================
- Coverage     45.54%   45.38%   -0.17%
  Complexity      495      495
============================================
  Files           129      129
  Lines          2470     2479       +9
  Branches        291      292       +1
============================================
  Hits           1125     1125
- Misses         1256     1265       +9
  Partials         89       89

Impacted Files
Coverage Δ

...eedu/address/logic/parser/ExportCommandParser.java
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 72b26d5...7046e82. Read the comment docs.


chore: :wrench: add Marton to the pyproject.toml file

Description
Closes #540
Reviewer Focus

This PR needs a quick review.

Yayy 🥳 @martonvago doesn't want his work mail instead of gmail here?

Ah, that's a good question. I use that gmail for work, so I think it works!


WIP: Fake SFU Example

This PR includes a change in the heroku demo to slice the MCU in multiple canvas elements.
Let's see if we want to port the code within the SDK or not.

Closing since we have other demos and the branch is too old.


core.js:7187 ERROR SyntaxError: Unexpected token E in JSON at position 0     at JSON.parse ()     at rxp-hpp.js:300

I'm testing my payment form and when I try to test a card that returns a 200 auth code, I receive the following error:
core.js:7187 ERROR SyntaxError: Unexpected token E in JSON at position 0
at JSON.parse ()
at rxp-hpp.js:300
It's because the message received from GlobalPayments is not in the correct format.

Came across the same issue. Here is my test case:

Put something in the far future like 11/44 for the expiry date for the credit card.
Using 4242 4242 4242 4242 for test cards.

The server returns "Error: 509Message: Expiry date invalid" as a string instead of a valid JSON, so the error happens when doing JSON.parse(event.data).
I'll have to do some modifications to the library as a hack, but the maintainer should fix it ASAP.

Did anyone find a solution to this? Im hitting this problem

Is this ever going to be addressed? I'm having the same issue

As am I - not a fan of modifying third party JS files as it means future updates are risky... but that's where we are I guess.

As am I - not a fan of modifying third party JS files as it means future updates are risky... but that's where we are I guess.

I had exactly same issue then at line 300 please include the following code
try {
				var test = event.data;

				console.log("test"+event.data);  //Here the log can print the issue and you can handle it.
				}

catch (err) {
console.log(err);
return;
}


Add license file

Ooh, I didn't catch that. I'll revise this PR. I'm looking to standardize all of our repos to have the license in LICENSE, without an extension (although it is definitely plain text).
README is a separate issue and can be in whatever format the project desires; however, I've seen README.md as the most popular option within IBM (as in this repo). And FWIW, I'd like to stick to Markdown for all of our repos (although, I personally prefer RestructuredText, I think it's far more important that we standardize on one solution).
However, if you meant "LICENSE" instead of "README", then I think the answer is more interesting :) GitHub actually documents one convention and implements another. GitHub's documentation on the topic suggests LICENSE.txt, and GitHub's licensing tool creates a LICENSE file (example repo, specifically to illustrate the result).


Help. I cannot install PiVPN on my Raspberry pi 4 (4gb)

when I run this "curl -L https://test.pivpn.io | TESTING= bash"
I get this error
Checking for git... already installed!
:::    Checking for tar... already installed!
:::    Checking for wget... already installed!
:::    Checking for curl... already installed!
:::    Checking for grep... already installed!
:::    Checking for dnsutils... not installed!
:::    Checking for whiptail... already installed!
:::    Checking for net-tools... already installed!
:::    Checking for bsdmainutils... already installed!
:::    Checking for dhcpcd5... already installed!
:::    Checking for iptables-persistent... not installed!
:::    Failed to install dnsutils!
:::    Failed to install iptables-persistent!
pi@raspberrypi:~ $

I should close this issue because It doesn’t really follow the issue template, however, the console output is there so... what’s the output of sudo apt-get —yes --no-install-recommends install dnsutils iptables-persistent?

I got it to work by following wherever in this blog https://blog.mxard.com/persistent-iptables-on-raspberry-pi-raspbian
PS: I'm very sorry if I broke any rules this is my first post in this website and I have no clue what im doing.


monorepo support

Does it support monorepo? If not, can you do that?

Hello!
git-cliff supports monorepos via --include-path and --exclude-path arguments.You can specify them using args input variable.
If this is not helpful for you, please give more details about your use case and consider submitting an issue to the git-cliff repository.


Return concrete types instead of interfaces

This is more idiomatic in Go. Also it saves the caller the need to typecast the returned object of calling viaproxy.Wrap to a viaproxy.Conn and thus giving automatically access to methods like ProxyAddr. And given that viaproxy.Conn implements the net.Conn interface, it can be passed as an argument to any other methods that accept a net.Conn.

I'm not sure if I should do the same with the viaproxy.Listen function, and return a net.TCPListener instead (proxy protocol only works for TCP connections).


Support multiple lists for one type of content

I'd like to be able to:
content/
    post/
        ...
layouts/
    post/
        ...
indexes/
    post/
        archive.html
        active.html

The intention here is to have an archive page of all posts and an active page of only active posts, say, the 5 latest posts.

I ran into this exact problem yesterday, but followed @spf13 's advice and figured out a work around.
Here's the post about it: https://fn.lc/post/hugo-multiple-lists/#multiple-list-views

Handled by https://github.com/spf13/hugo/issues/2828

@bep
I don't see how it's handled by #2828
Can you please explain, how is that related


Add support for regional Splatfest

This PR aims to add support for regional Splatfests
The newest Splatfest is split between JP and EU/NA/AP, so this PR aims at fixing everything in s3.ink so that it's not US-specific
Draft while everything gets updated

Upcoming Splatfest

Active Splatfest (Website)

Active Splatfest (Screeenshot)

Recent Splatfest

Important Notes
This PR assumes regional Splatfests run at the same time, which in this case they are. If, in future, this is not the case, a lot needs to be amended including schedule fetching due to the possibility of having both normal and fest schedules happing at the same time

Thank you @slushiegoose!! This looks fantastic. I just made some tweaks to fix the social status generators.


BrAPI Pedigree Viewer

Request to look into BrAPI Pedigree Viewer for the SoyBase parentage data.
Current SoyBase parentage webpage is located here.
A few things to consider:

What are the pedigree viewer dependencies?
What is the file format required for the viewer?
Can the pedigree be drawn quickly for the user or does a pedigree diagram need to be drawn and saved for users?
The example BrAPI viewer has a user name and password. Is the username and passwd required?

It appears we will have to have a germplasm BRAPI compliant end point.  That is what the pedigree viewer uses.  It seems that a service could be made that utilizes a text file or a MySQL DB.  If a text file is used there would be some lag reading it into a hash.  We would have to distinguish not only parent child relationships but also "siblings".  I don't have a sibling relationship in the database.  You can identify those through the comments field.  I usually put "reselection" to denote a sibling.  We would probably want to change the table to include a "sibling" field.  This would have to be reflexive of course.


Add initial alternative totp method

This is the PR for #33.
I had to pass along the name and folder as well, since it's needed for the code call.
When using the --action=menu option it's a a tiny bit (but noticeable in comparison) slower before the menu is shown, do you think that's bad/unwanted/too noticeable?

Again, I like the general idea, but the concrete implementation was missing some things (for example, it now contained some logic to call rbw, but not all of it). So I've taken the liberty to implement the changes myself, but - again - credit you as co-author. I do hope this is okay for you, but if not, please tell me 🙂

I like your implementation better!


Feature backward compatible simple test format

With this feature, there is no need to migrate exercise format, and we can still write simple exercises in a clean, concise way
@Charlyzzz @faloi

@faloi please review and merge, and push a new version


setDuration

setDuration() needs a positive and greater than Zero number, however, all durations are negative or zero.
Android studio error:
code =  setDuration(Snacky.LENGTH_LONG)
error =

Must be one of: BaseTransientBottomBar.LENGTH_INDEFINITE, BaseTransientBottomBar.LENGTH_SHORT, BaseTransientBottomBar.LENGTH_LONG or value must be ≥ 1 (was -2)

Hi! Thank you for your filling an issue.
I don't get you point. Is this an error you get during build? Or just AndroidStudio complaining about a wrong value? I just found out that an annotation is wrong, but that should not fail compiling... I will correct the annotation and make an hotfix release soon.


Resulting sparsity is 0 for all the layers

Hi,
I have tried to prune two different models, one for image regression (resnet) and another for segmentation (unet, resnet backbone).
Both after finished with pruning return 0 for all layers with tensor_sparsity(). Tried running the beans tutorial and there the pruning worked.
Here is my recipe for the segmentation:
# Epoch hyperparams
stabilization_epochs: 1.0
pruning_epochs: 9.0
finetuning_epochs: 5.0

# Learning rate hyperparams
init_lr: 0.0001
final_lr: 0.00005

# Pruning hyperparams
init_sparsity: 0.05
final_sparsity: 0.9

# Stabalization Stage
training_modifiers:
  - !EpochRangeModifier
    start_epoch: 0.0
    end_epoch: eval(stabilization_epochs + pruning_epochs + finetuning_epochs)

  - !SetLearningRateModifier
    start_epoch: 0.0
    learning_rate: eval(init_lr)

# Pruning Stage
pruning_modifiers:
  - !LearningRateFunctionModifier
    init_lr: eval(init_lr)
    final_lr: eval(final_lr)
    lr_func: cosine
    start_epoch: eval(stabilization_epochs)
    end_epoch: eval(stabilization_epochs + pruning_epochs)

  - !GlobalMagnitudePruningModifier
    init_sparsity: eval(init_sparsity)
    final_sparsity: eval(final_sparsity)
    start_epoch: eval(stabilization_epochs)
    end_epoch: eval(stabilization_epochs + pruning_epochs)
    update_frequency: 0.5
    params:
        - 'encoder._conv_stem.weight'
        - 'encoder._blocks.0._depthwise_conv.weight'
        - 'encoder._blocks.0._se_reduce.weight'
        - 'encoder._blocks.0._se_expand.weight'
        - 'encoder._blocks.0._project_conv.weight'
        - 'encoder._blocks.1._expand_conv.weight'
        - 'encoder._blocks.1._depthwise_conv.weight'
        - 'encoder._blocks.1._se_reduce.weight'
        - 'encoder._blocks.1._se_expand.weight'
        - 'encoder._blocks.1._project_conv.weight'
        - 'encoder._blocks.2._expand_conv.weight'
        - 'encoder._blocks.2._depthwise_conv.weight'
        - 'encoder._blocks.2._se_reduce.weight'
        - 'encoder._blocks.2._se_expand.weight'
        - 'encoder._blocks.2._project_conv.weight'
        - 'encoder._blocks.3._expand_conv.weight'
        - 'encoder._blocks.3._depthwise_conv.weight'
        - 'encoder._blocks.3._se_reduce.weight'
        - 'encoder._blocks.3._se_expand.weight'
        - 'encoder._blocks.3._project_conv.weight'
        - 'encoder._blocks.4._expand_conv.weight'
        - 'encoder._blocks.4._depthwise_conv.weight'
        - 'encoder._blocks.4._se_reduce.weight'
        - 'encoder._blocks.4._se_expand.weight'
        - 'encoder._blocks.4._project_conv.weight'
        - 'encoder._blocks.5._expand_conv.weight'
        - 'encoder._blocks.5._depthwise_conv.weight'
        - 'encoder._blocks.5._se_reduce.weight'
        - 'encoder._blocks.5._se_expand.weight'
        - 'encoder._blocks.5._project_conv.weight'
        - 'encoder._blocks.6._expand_conv.weight'
        - 'encoder._blocks.6._depthwise_conv.weight'
        - 'encoder._blocks.6._se_reduce.weight'
        - 'encoder._blocks.6._se_expand.weight'
        - 'encoder._blocks.6._project_conv.weight'
        - 'encoder._blocks.7._expand_conv.weight'
        - 'encoder._blocks.7._depthwise_conv.weight'
        - 'encoder._blocks.7._se_reduce.weight'
        - 'encoder._blocks.7._se_expand.weight'
        - 'encoder._blocks.7._project_conv.weight'
        - 'encoder._blocks.8._expand_conv.weight'
        - 'encoder._blocks.8._depthwise_conv.weight'
        - 'encoder._blocks.8._se_reduce.weight'
        - 'encoder._blocks.8._se_expand.weight'
        - 'encoder._blocks.8._project_conv.weight'
        - 'encoder._blocks.9._expand_conv.weight'
        - 'encoder._blocks.9._depthwise_conv.weight'
        - 'encoder._blocks.9._se_reduce.weight'
        - 'encoder._blocks.9._se_expand.weight'
        - 'encoder._blocks.9._project_conv.weight'
        - 'encoder._blocks.10._expand_conv.weight'
        - 'encoder._blocks.10._depthwise_conv.weight'
        - 'encoder._blocks.10._se_reduce.weight'
        - 'encoder._blocks.10._se_expand.weight'
        - 'encoder._blocks.10._project_conv.weight'
        - 'encoder._blocks.11._expand_conv.weight'
        - 'encoder._blocks.11._depthwise_conv.weight'
        - 'encoder._blocks.11._se_reduce.weight'
        - 'encoder._blocks.11._se_expand.weight'
        - 'encoder._blocks.11._project_conv.weight'
        - 'encoder._blocks.12._expand_conv.weight'
        - 'encoder._blocks.12._depthwise_conv.weight'
        - 'encoder._blocks.12._se_reduce.weight'
        - 'encoder._blocks.12._se_expand.weight'
        - 'encoder._blocks.12._project_conv.weight'
        - 'encoder._blocks.13._expand_conv.weight'
        - 'encoder._blocks.13._depthwise_conv.weight'
        - 'encoder._blocks.13._se_reduce.weight'
        - 'encoder._blocks.13._se_expand.weight'
        - 'encoder._blocks.13._project_conv.weight'
        - 'encoder._blocks.14._expand_conv.weight'
        - 'encoder._blocks.14._depthwise_conv.weight'
        - 'encoder._blocks.14._se_reduce.weight'
        - 'encoder._blocks.14._se_expand.weight'
        - 'encoder._blocks.14._project_conv.weight'
        - 'encoder._blocks.15._expand_conv.weight'
        - 'encoder._blocks.15._depthwise_conv.weight'
        - 'encoder._blocks.15._se_reduce.weight'
        - 'encoder._blocks.15._se_expand.weight'
        - 'encoder._blocks.15._project_conv.weight'
        - 'encoder._conv_head.weight'
        - 'decoder.blocks.0.conv1.0.weight'
        - 'decoder.blocks.0.conv2.0.weight'
        - 'decoder.blocks.1.conv1.0.weight'
        - 'decoder.blocks.1.conv2.0.weight'
        - 'decoder.blocks.2.conv1.0.weight'
        - 'decoder.blocks.2.conv2.0.weight'
        - 'decoder.blocks.3.conv1.0.weight'
        - 'decoder.blocks.3.conv2.0.weight'
        - 'decoder.blocks.4.conv1.0.weight'
        - 'decoder.blocks.4.conv2.0.weight'
        - 'segmentation_head.0.weight'
    leave_enabled: True

# Finetuning Stage
finetuning_modifiers:
  - !LearningRateFunctionModifier
    init_lr: eval(init_lr)
    final_lr: eval(final_lr)
    lr_func: cosine
    start_epoch: eval(stabilization_epochs + pruning_epochs)
    end_epoch: eval(stabilization_epochs + pruning_epochs + finetuning_epochs)

Only things that I add to my regular code are:
manager = ScheduledModifierManager.from_yaml('sparsify_recipe.yaml')
optimizer = manager.modify(model, optim, steps_per_epoch=len(train_loader))

and at the end,
manager.finalize(model)

After that I save model with torch.save() as tar file. After loading and running tensor_sparsity() it returns 0.000 for all the layers.
Not sure if this is a bug necessarily, maybe I do something wrong that I am not aware of.
Thank you.

What is your model definition and what the intergration to the training code looks like?

Thank you for your reply, here is my code:
train_batch_size = 64
val_batch_size = 1

shape = (224, 224)

train_loader = DataLoader(
    train_dset,
    batch_size=train_batch_size,
    shuffle=True,
    num_workers=6)

lr = 0.0005

model = smp.Unet('efficientnet-b0', activation=None, encoder_weights=None)
model = model.cuda()

optim = torch.optim.Adam(model.parameters(), lr=lr)

epochs = 50
no_batches_val = math.ceil(val_size / val_batch_size)
loss = BCEDiceLoss()
no_batches_train = math.ceil(train_size / train_batch_size)

checkpoint = torch.load('efbn0_224_sparsify_test(14).tar')
model.load_state_dict(checkpoint['model_state_dict'])

manager = ScheduledModifierManager.from_yaml('sparsify_recipe.yaml')
optimizer = manager.modify(model, optim, steps_per_epoch=len(train_loader))
train_loss_history = []
train_iou_history = []
for epoch in tqdm(range(manager.max_epochs)):

    model = model.cuda().train()

    for (imgs, labels, _) in tqdm(train_loader):
        optim.zero_grad()

        imgs, labels = imgs.cuda(), labels.cuda()

        out = model(imgs)
        batch_train_loss = loss(out, labels)

        batch_train_loss.backward()
        optim.step()

manager.finalize(model)

torch.save({
            'epoch': epoch,
            'model_state_dict': model.state_dict(),
            },  'efbn0_224_sparse_final(' + str(epoch) + ').tar')

Hi @Tramtadama, I'll take a look into this and see if I can reproduce. A few questions:

Could you clarify what dataset you are using? I see you're passing train_dset to a DataLoader but your code doesn't include the definition
Could you also include the definition for BCEDiceLoss()?
Is this example supposed to be for image segmentation or classification? The model you're loading seems to be for segmentation, but you're calculating loss on labels

Hi @Satrat, thank you for you efforts. The task is binary segmentation. The "label" is 224x224 image mask of 0's and 1's.
Here is the definition of the loss function:
import torch
from torch import nn

def f_score(pr, gt, beta=1, eps=1e-7, threshold=None, activation='sigmoid'):
    """
    Args:
        pr (torch.Tensor): A list of predicted elements
        gt (torch.Tensor):  A list of elements that are to be predicted
        beta (float): positive constant
        eps (float): epsilon to avoid zero division
        threshold: threshold for outputs binarization
    Returns:
        float: F score
    """

    if activation is None or activation == "none":
        activation_fn = lambda x: x
    elif activation == "sigmoid":
        activation_fn = torch.nn.Sigmoid()
    elif activation == "softmax2d":
        activation_fn = torch.nn.Softmax2d()
    else:
        raise NotImplementedError(
            "Activation implemented for sigmoid and softmax2d"
        )

    pr = activation_fn(pr)

    if threshold is not None:
        pr = (pr > threshold).float()

    tp = torch.sum(gt * pr)
    fp = torch.sum(pr) - tp
    fn = torch.sum(gt) - tp

    score = ((1 + beta ** 2) * tp + eps) \
            / ((1 + beta ** 2) * tp + beta ** 2 * fn + fp + eps)

class DiceLoss(nn.Module):
    __name__ = 'dice_loss'

    def __init__(self, eps=1e-7, activation='sigmoid'):
        super().__init__()
        self.activation = activation
        self.eps = eps

    def forward(self, y_pr, y_gt):
        return 1 - f_score(y_pr, y_gt, beta=1., eps=self.eps, threshold=None, activation=self.activation)```

class BCEDiceLoss(DiceLoss):
    __name__ = 'bce_dice_loss'

    def __init__(self, eps=1e-7, activation='none'):
        super().__init__(eps, activation)
        self.bce = nn.BCEWithLogitsLoss(reduction='mean')

    def forward(self, y_pr, y_gt):
        dice = super().forward(y_pr, y_gt)
        bce = self.bce(y_pr, y_gt)
        return dice + bce

Hope this helps with the reproduction, if there is anything more that you need please tell me. The dataset is private so I do not know if providing the definition helps? It is an image of a car with license plate and corresponding binary mask where the area of license plate are 1's and everything else is 0, there are also images where there is no license plate so corresponding mask is just 0's.
Thank you!

Fixed the optimizer mistake, now its working. Sorry for the inconvenience.


next/previous links have always href="./"

this line does something that is not expected: https://github.com/pietroppeter/nimibook/blob/main/src/nimibook/resources/templates/document.mustache#L189

😫 I never tried running it on a server, only locally :/
Can it have something to do with entry.url not doing the same locally as online?

I have no clue why it is not working as expected. I cannot see why entry.url should work any different locally vs server (and anyway it is used in rendering the toc, and in that case it works fine).
To me the implementation looks fine but there is defintely something not working correctly.

Hmm it seems to be something wrong with the Github CI then because when I build it on Netlify here the buttons suddenly work. And I'm just doing nim r getting_started.nim build 🤔

Yes. Nice work on Netlify!

I tested with a commit to see if this issue disappears (I see next/previous working fine on SciNim/getting-started) but it still there. I guess there is some difference in set up of nimibook or CI.

So strange... The only real difference I can see between the build scripts is that you use nimble genbook and we do nim c getting_started.nim, getting_started init, getting_started build

Another notable difference is that you have a bunch of config.nims files changing paths while we don't.

ok, after struggling a little with adding plausible analytics directly in book api I know understand where this issue is coming from and I was also able to fix it. The issue in the CI that does rebase gh-pages on main and gh-pages has a lot of cruft that stays there (every file removed in main is still present in hg-pages). It does not help that build mechanism will not replace the document.mustache if one was commited. The simple fix was to remove artificially document.mustache in gh-pages and push another commit to rerun CI.
Better fixes planned:

#33
#32


[FAQ] How to add tooltips to the legend of a chart?

As shown in the following figure, the text of the legend is omitted. I want to add prompt information to them to display the complete text. How should I operate?

https://visactor.com/vchart/faq/92-How to add prompt information to the legend


Added auto coursor move when typing.

Added new functionality
Summary of Changes

Cursor will automaticly move to next input when typing letters
Cursor will automaticly move back to previous input when pressing backspace and when current input is empty

Still sorry for that, but I don't know how to make it as a function wich will be called after every new rebus, so temporarly I just added it to registerListeners function.

Pull Request Test Coverage Report for Build 165

2 of 8 (25.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-8.4%) to 82.353%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/js/app.js
2
8
25.0%

Totals

Change from base Build 164:
-8.4%

Covered Lines:
99

Relevant Lines:
115

💛  - Coveralls


make grid invisible

Hi. I would like to make the grid invisible. I tried it with the standard Makie commands in the italian map:
let
	it_states = Downloads.download("https://github.com/openpolis/geojson-italy/raw/master/geojson/limits_IT_provinces.geojson")
	geo = GeoJSON.read(read(it_states, String))

	fig = Figure()
	ga = GeoAxis(fig[1, 1]; dest = "+proj=ortho +lon_0=12.5 +lat_0=42", lonlims=(12, 13), latlims = (40, 44), xgridvisible=false, ygridvisible=false, xticklabelsvisible=false, yticklabelsvisible=false, xticksvisible=false, yticksvisible=false)
	poly!(ga, geo; strokecolor = :blue, strokewidth = 1, color = (:blue, 0.5), shading = false);
	datalims!(ga)

	fig
end

it works for ticks and labels but not for the grid.

I want to mention that hidedecorations!(ga) is working.

I have the same issue. Would like to know if a solution exists.

I use this to hide the grid and labels
hidedecorations!(ga)
hidespines!(ga)

Should work now, the API is significantly different as well.


Add empty string default to prevent errors when key is missing.

Background
We saw errors in our environment from these rules because accessKeyId is an optional field that does not appear in all cloudtrail logs (aws_cloudtrail_useraccesskeyauth) and not all Okta logs have the displayName field under the target key (okta_sso_to_aws).
Changes
A null string default was added to event.deep_walk() in these rules.
Testing
Unit tests were added for when these keys are missing.
FYI @arielkr256

@nickchap good catch, thank you.  These have been addressed here https://github.com/panther-labs/panther-analysis/pull/1311


Added sections to conditional hooks page

5 new hooks have to be documented
The new sectioning makes it easy to see what the hooks are for
Placeholders opnly added for when, unless, some, every

ekryski wrote"Yup looks good! Go for it!"


Website not live yet

Website: https://csa.idsl.me is not active yet?
Also the info has some weird characters?
> library(IDSL.CSA)
> IDSL.CSA_workflow("Address of the CSA parameter spreadsheet")

Initiated testing the IDSL.CSA workflow spreadsheet consistency!
The IDSL.CSA workflow spreadsheet not found! It should be an Excel file with .xlsx extention!
Please visit    https://csa.idsl.me    for instructions!!!
>

The website is down and the communication message will no longer be included in future releases.
You can download IDSL.CSA parameter spreadsheet. and fill it with the required processing parameters. When you are ready to run the analysis, you can provide the address of the CSA parameter spreadsheet on your local drive as the input to the IDSL.CSA_workflow function. The packages will then read in the parameter values from the spreadsheet and use them in the analysis.
Those weird characters are actually used to adjust R command colors to highlight error messages, and they have worked fine with Ubuntu and Windows systems.
cat("\033[92mEverything is OK!!\033[0m\n")
cat("\033[91mError: Something went wrong!\033[0m\n")


better whitelist generation

Right now whitelist generation is a manual process. I've tried in the past to write some code which gives a better piecemeal approach at generating these, but that's been inconclusive at best.
What I've been thinking about is offering a way to read browser history and group up what certs are really used. The process would look something like:

Scan browser history (Chrome, Firefox, Safari, Edge) and group up domain names
Iterate over the domains and find their respective certs
Find what intermediates/CA's are signing each and generate a whitelist with just those.

https://github.com/adamdecaf/cert-manage/commit/2e9d62b921a180b6e54bce8d3368f230bb030d1f

I've started a bit of work over in https://github.com/adamdecaf/browsehist, but it's requiring CGO for now, which will make cross-compiling harder..


Add some example YAML files to README

It would be helpful if there were some actual YAML examples in the README to demonstrate what an example YAML file should look like, and how that gets converted into Sphinx documentation.

Thank you for feedback. I've included example in README. Also I encourage to take a look at tests: https://github.com/Jakski/sphinxcontrib-autoyaml/tree/4d8113af77978c54d38092d48fa808cf3e7330ba/tests/examples/output
They cover most use cases.

Looks great! Closing issue as completed.


Sophos Anti-Virus software reports ransomware activities

We use GitHub Issues for bugs.
If you have a non-bug question, ask on Stack Overflow: http://stackoverflow.com/questions/tagged/react-native
If you have a feature request, post it on Product Pains: https://productpains.com/product/react-native/
--- Please use this template, and delete everything above this line before submitting your issue ---
Description
After installing react-native bundle, Sophos Anti-Virus software keeps prompting for ransomware warning in /usr/local/bin/node.
Please see attached screenshot.

Reproduction
[FILL THIS OUT: Try to reproduce your bug on rnplay.org and provide a link. If you can't reproduce the bug on rnplay.org, provide a sample project.]
Solution
[FILL THIS OUT: What needs to be done to address this issue? Ideally, provide a pull request with a fix.]
Additional Information

React Native version: [FILL THIS OUT: Does the bug reproduce on the latest RN release?]
Platform: [FILL THIS OUT: iOS, Android, or both?]
Operating System: [FILL THIS OUT: MacOS, Linux, or Windows?]

We're cutting down on the number of outstanding issues, in order to allow us to focus. I'm closing this issue because it has been open for over 60 days with no activity. If you think it should still be opened let us know why. PRs are always welcome.

Thanks for looking into this. It was the company's securities rules that caused this.


css: Scroll on body instead of .app.

This removes horizontal position dependence of fixed positioned elements, navbar and compose, from the scroll bar.
We make body scrollable and make navbar and compose sticky so that they are inside the flow of the document and thus do not depend on the scrollbar width.
It would have been better to switch to html to fix the dependence on scrollbar but switching to html changes the value of offset().top for all elements since the value of offset depends on document height. This alone introduces hard to fix bugs since we have such carefully curated conditions and calculations for every scenario. Also, since document has the power to scroll now, changes to # can introduce unpredictable scrolling effects.
So, maybe we can switch to html in a rewrite, but right now, switching to body is a lower risk change than switching to html.
Discussion: https://chat.zulip.org/#narrow/stream/9-issues/topic/users.20list.20overlaps.20compose.20box

Test-deploying this on chat.zulip.org so we can get some burn-in there.

Thanks! I found the following issues when playing with this:

Expanded compose box breaks if you start vertically resizing the window.
I'm seeing weird space below the compose box when switching into a view that has blank space at the bottom. (It fixes itself if you then resize the window.)

Tested and didn't see any problems:

No double scroll bars for me (on a Mac)
Information overlays shadow
Invite more users (I think it would be nicer if it didn't shift down at all, but this is the same as I see on Zulip Cloud.)
Recent conversations (vertical lines are gone)

Thanks! Fixed both the issues.

Looks good to me when I retest!

Pushed to resolve conflicts.

I extracted the HTML adjustments to index.html to https://github.com/zulip/zulip/pull/25689, since those seem safe to merge into main and will help shrink the complexity of this PR.

Closing in favour of  #25697


Updated bash completions with missing commands.

Hi.
I tried my best to add missing commands and correct some of them.
For commands that need --targets, I have the idea to put the result in a cache file associated with the current session, like that is faster on the next usage of the --targets. I did not add an option to delete the file.
And I also added a test before to check if the previous keywords are a flag and not need completion like the --targets or --rerun flag to avoid losing the current command completion.
I tested on macOS ZSH and Bash, and on Ubuntu with Bash. It seems to works fine.
Let me know what you think about the modifications.

CLA signed by all contributors.

I updated the commit message and added a release note marker and merged it. Thank you for the updates!