vim+screen: X11 server change disables X11 copy/paste

Vim cached X11 primary selection ownership information is obsolete when using vim with GNU screen and changing the connected X server instance.
ex. if local machine is rebooted, local X server is restarted, ssh X11 forwarding is restarted, or connecting from a different local machine
Unable to  access X11 primary selection unless vim is restarted
ex. can't copy to primary X11 clipboard
To recreate:
at local xterm
rmt=raspberrypi
DISPLAY=:0 ssh -fYp 13887 $rmt 'xterm -T remote_vim -e screen -m -S remote_vim vim'
at remote_vim
:.!echo -e "echo one   one   one\necho two   two   two\necho three three three"
at local xterm
kill ssh X11 forwarding, screen session survives
kill -INT $(ps ax | grep remote_vim | grep -vw grep | cut -d " " -f 1)
DISPLAY=:0 ssh -fYp 13887 $rmt 'xterm -T remote_vim -e screen -r remote_vim'
vi # in local terminal, not at remote_vim
enter insert mode
at remote_vim
select line
at local xterm
paste line from remote_vim, this will fail to work correctly

duplicate of #844

Vim cached X11 primary selection ownership information is obsolete when using vim with GNU screen and changing the connected X server instance.

Vim caches neither primary nor pasteboard selection ownership; it only asks the X11 server which is known to Vim at startup for selection ownership every time it needs it, which implies that Vim has to establish and keep connection with the server before and when Vim's copy and paste through X11 selection mechanism is carried out.  Therefore, Vim won't be able to work as you expected when the server is restarted since the established connection between those particular instances of Vim and the X11 server has already lost then.
As another side of the same coin, by restarting Vim after rebooting the local system and restarting the X11 server, Vim works as you reported since a new connection between the new instance of Vim and that of the X11 server is established at the startup of the Vim.

:xrestore
E492: not an editor command

it hasn't been merged yet.

@nuko8 I believe Vim tries to reset the clipboard connection if it notices that the X-Connection wen't down. See #203 and patch 7.4.523

So....... what do I do with this ?

@chrisbra  Ah, I was wrong.  Somehow I forgot that difference between primary and pasteboard.  Sorry about missing that and thank you for correcting me.

Let me try again.   What's the best way to handle this ?

do not trouble yourselves further I've used the commits referenced in #844 and prepared an equivalent patch for vim-8.1.1 to add the xrestore facility.  It seems to be working well.

do not trouble yourselves further I've used the commits referenced in
#844 and prepared an equivalent patch
for vim-8.1.1 to add the xrestore facility.  It seems to be working well.

I am wondering why :xrestore is needed.  Is that because the already built-in
mechanism to reconnect does not always work?  Or because you intentionally want
to connect to another X server?
--
INSPECTOR END OF FILM: Move along.  There's nothing to see!  Keep moving!
[Suddenly he notices the cameras.]
INSPECTOR END OF FILM: (to Camera) All right, put that away sonny.
[He walks over to it and puts his hand over the lens.]
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

As far as I know, there is no such mechanism, at least not in the release versions up to 8.1.1.  I have always had to restart it if the X server was restarted or changed for whatever reason.  There have been patches and reports involving this going back several years, but to my understanding none have been merged as of 8.1.1.  I know there has never been any functioning reconnect in releases from 7.1 onwards.

The entry for #844 mentions conflicts which need to be resolved, which may explain something about why a fix for X reconnection was not available.

I've found the automatic reconnection mechanism code, or what appears to be it.  The ex_xrestore() function (from the #844 commits) doesn't seem to actually do much more than provide a new display name and set a display reset flag (xterm_dpy_was_reset=1).  It looks like all the rest of the code except for the X error handler "x_IOerror_handler() would have to be working in order for the xrestore command to be working, which it is (when patched in).   It would seem that the x_IOerror_handler() function is not being called for some reason, but I have yet to verify this.  If that turns out to be the case, it would explain the issues I'm seeing.

And of course, as I should have noted more clearly earlier, the xrestore command is useless in .vimrc anyway, but I should patch it further so it becomes a nop if called from .vimrc.   I really need to get the automatic X reconnection to work properly though.

It turns out that the error handler x_IOerror_handler() is in fact being invoked, but the XtOpenDisplay() display reopening (which is called from setup_term_clip, from may_restore_clipboard) fails after the x_IOerror_handler() is invoked, but not before.  Enabling app_context destruction in may_restore_clipboard() does not prevent this failure.
I am now trying to find out why the XtOpenDisplay() display reopen fails.

Is there a way to attach a file ?

diff -ru vim81/src/os_unix.c vim81.new/src/os_unix.c
--- vim81/src/os_unix.c	2018-05-12 11:42:33.000000000 -0400
+++ vim81.new/src/os_unix.c	2018-12-03 21:28:06.119161750 -0500
@@ -7468,6 +7468,8 @@
 	{
 	    xterm_dpy = XtOpenDisplay(app_context, xterm_display,
 		    "vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);
+            if(!xterm_dpy)
+		xterm_dpy_was_reset = TRUE;
 #if defined(HAVE_SETJMP_H)
 	    mch_endjmp();
 #endif

That should do it.

Doesn't this mean that if the XtOpenDisplay() always fails, we keep trying?
That is not good, because it may be a slow operation.
Perhaps we should add a flag that it once worked, instead of just a flag to remember it failed.

As far as I can tell, XtOpenDisplay() only fails for an extremely short timeout period  immediately following a server disconnection.  I'm using that patch extensively now, and am no longer having any difficulties changing servers at all.  I don't get any noticeable delays whatsoever.

As far as I can tell, XtOpenDisplay() only fails for an extremely
short timeout period  immediately following a server disconnection.
I'm using that patch extensively now, and am no longer having any
difficulties changing servers at all.  I don't get any noticeable
delays whatsoever.

Sure, it fixes your problem.  But does it create a new problem for other
situations?  The Xserver does not have to be local, it can be remote,
which means communication is much slower, and can involve timeouts.
We need to prevent from bad things happening if XtOpenDisplay() fails
for another reason.
--
Why I like vim:

I like VIM because, when I ask a question in this newsgroup, I get a
one-line answer.  With xemacs, I get a 1Kb lisp script with bugs in it ;-)

/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

There is that consideration.  I'm afraid I'm not a member of the project though....:-)   That said, what would you like to see done with this ?

For the record, I tried the patch in my local installation. I usually run Vim without an available X-Server inside a putty terminal. The patch makes Vim unresponsive.
How about something along this:
diff --git a/src/os_unix.c b/src/os_unix.c
index 4d902d939..1537ed951 100644
--- a/src/os_unix.c
+++ b/src/os_unix.c
@@ -1704,9 +1704,12 @@ x_IOerror_handler(Display *dpy UNUSED)
     static void
 may_restore_clipboard(void)
 {
-    if (xterm_dpy_was_reset)
+    static int count = 0;
+
+    if (xterm_dpy_was_reset && ++count % 100 == 0)
     {
        xterm_dpy_was_reset = FALSE;
+       count = 0;

 # ifndef LESSTIF_VERSION
        /* This has been reported to avoid Vim getting stuck. */
@@ -7498,6 +7501,12 @@ setup_term_clip(void)
        {
            xterm_dpy = XtOpenDisplay(app_context, xterm_display,
                    "vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);
+           if(!xterm_dpy)
+           {
+               xterm_dpy_was_reset = TRUE;
+               if (p_verbose > 0)
+                   verb_msg((char_u *)_("Failed to open X Display"));
+           }
 #if defined(HAVE_SETJMP_H)
            mch_endjmp();
 #endif

I just gave this next patch a try by running it under screen, killing the X server, and restarting it.  The first several copy/paste attempts fail.   Eventually one succeeds, and the rest work properly after that.  I'll need to see why that might be.   I tried the earlier "two-line" patch from the console with no X server running, and did not see any lack of response.  I don't know for sure if that has any bearing on what you experience running inside a putty terminal without an X server.  I don't have putty installed, but I'm thinking of installing it and trying to use vim from a putty terminal as you did to see what happens with the "two line" patch there..

Can you give me an example of what you are doing with putty so I can try to recreate it ?

I am also thinking that changing 100 to a smaller number might help with this larger patch.

I am just using putty to ssh to my linux vmware for developing Vim. And just starting Vim with your patch caused it to become unresponsive. I couldn't even enter :q<cr>. I assumed this was because Vim was constantly trying to reconnect to the X-Server.

I am also thinking that changing 100 to a smaller number might help with this larger patch.

Yeah, not sure what a good compromise would be. Perhaps 50?

Seems to work perfectly for me at 50.   Here is the latest:
diff -ru vim81/src/os_unix.c vim81.new/src/os_unix.c
--- vim81/src/os_unix.c	2018-05-12 11:42:33.000000000 -0400
+++ vim81.new/src/os_unix.c	2018-12-07 17:50:41.570243632 -0500
@@ -1712,9 +1712,12 @@
     static void
 may_restore_clipboard(void)
 {
-    if (xterm_dpy_was_reset)
+    static int count = 0;
+
+    if (xterm_dpy_was_reset && ++count % 50 == 0)
     {
 	xterm_dpy_was_reset = FALSE;
+	count = 0;

 # ifndef LESSTIF_VERSION
 	/* This has been reported to avoid Vim getting stuck. */
@@ -7468,6 +7471,12 @@
 	{
 	    xterm_dpy = XtOpenDisplay(app_context, xterm_display,
 		    "vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);
+	    if(!xterm_dpy)
+	    {
+		xterm_dpy_was_reset = TRUE;
+		if (p_verbose > 0)
+		    verb_msg((char_u *)_("Failed to open X Display"));
+	    }
 #if defined(HAVE_SETJMP_H)
 	    mch_endjmp();
 #endif

Seems to work perfectly for me at 50.   Here is the latest:
diff -ru vim81/src/os_unix.c vim81.new/src/os_unix.c
--- vim81/src/os_unix.c	2018-05-12 11:42:33.000000000 -0400
+++ vim81.new/src/os_unix.c	2018-12-07 17:50:41.570243632 -0500
@@ -1712,9 +1712,12 @@
     static void
 may_restore_clipboard(void)
 {
-    if (xterm_dpy_was_reset)
+    static int count = 0;
+
+    if (xterm_dpy_was_reset && ++count % 50 == 0)
     {
 	xterm_dpy_was_reset = FALSE;
+	count = 0;

 # ifndef LESSTIF_VERSION
 	/* This has been reported to avoid Vim getting stuck. */
@@ -7468,6 +7471,12 @@
 	{
 	    xterm_dpy = XtOpenDisplay(app_context, xterm_display,
 		    "vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);
+	    if(!xterm_dpy)
+	    {
+		xterm_dpy_was_reset = TRUE;
+		if (p_verbose > 0)
+		    verb_msg((char_u *)_("Failed to open X Display"));
+	    }
 #if defined(HAVE_SETJMP_H)
 	    mch_endjmp();
 #endif

We don't need two numbers, and 50 seems like a lot.
How about this instead:
--- ../../git/vim81/src/os_unix.c	2018-11-12 21:45:04.600729393 +0100
+++ os_unix.c	2018-12-08 14:06:36.902458131 +0100
@@ -1678,13 +1678,13 @@
/*

An X IO Error handler, used to catch terminal errors.
*/
-static int xterm_dpy_was_reset = FALSE;
+static int xterm_dpy_retry_count = 0;
static int
x_IOerror_handler(Display *dpy UNUSED)
{
xterm_dpy = NULL;

xterm_dpy_was_reset = TRUE;

xterm_dpy_retry_count = 5;  // Try reconnecting five times
x11_window = 0;
x11_display = NULL;
xterm_Shell = (Widget)0;
@@ -1704,9 +1704,9 @@
static void
may_restore_clipboard(void)
{

if (xterm_dpy_was_reset)

if (xterm_dpy_retry_count > 0)
{

xterm_dpy_was_reset = FALSE;

--xterm_dpy_retry_count;

ifndef LESSTIF_VERSION
/* This has been reported to avoid Vim getting stuck. */

@@ -7498,6 +7498,8 @@
{
xterm_dpy = XtOpenDisplay(app_context, xterm_display,
"vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);

  if (xterm_dpy != NULL)

  xterm_dpy_retry_count = 0;

#if defined(HAVE_SETJMP_H)
mch_endjmp();
#endif
--
Hacker: Someone skilled in computer programming (good guy).
Cracker: A hacker that uses his skills to crack software (bad guy).
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

This one fails with xterm_dpy_retry_count=5, but works well with xterm_dpy_retry_count=10.
Here is the latest:
diff -ru vim81/src/os_unix.c vim81.new/src/os_unix.c
--- vim81/src/os_unix.c	2018-05-12 11:42:33.000000000 -0400
+++ vim81.new/src/os_unix.c	2018-12-08 12:25:03.195929759 -0500
@@ -1686,13 +1686,13 @@
  */
 static int x_IOerror_handler(Display *dpy);
 static void may_restore_clipboard(void);
-static int xterm_dpy_was_reset = FALSE;
+static int xterm_dpy_retry_count = 0;

     static int
 x_IOerror_handler(Display *dpy UNUSED)
 {
     xterm_dpy = NULL;
-    xterm_dpy_was_reset = TRUE;
+    xterm_dpy_retry_count = 10; // Try reconnecting 10 times
     x11_window = 0;
     x11_display = NULL;
     xterm_Shell = (Widget)0;
@@ -1712,9 +1712,9 @@
     static void
 may_restore_clipboard(void)
 {
-    if (xterm_dpy_was_reset)
+    if (xterm_dpy_retry_count > 0)
     {
-	xterm_dpy_was_reset = FALSE;
+	--xterm_dpy_retry_count;

 # ifndef LESSTIF_VERSION
 	/* This has been reported to avoid Vim getting stuck. */
@@ -7468,6 +7468,8 @@
 	{
 	    xterm_dpy = XtOpenDisplay(app_context, xterm_display,
 		    "vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);
+	    if (xterm_dpy != NULL)
+		xterm_dpy_retry_count = 0;
 #if defined(HAVE_SETJMP_H)
 	    mch_endjmp();
 #endif

This one fails with xterm_dpy_retry_count=5, but works well with
xterm_dpy_retry_count=10.

Hmm, then how do you explain that retrying five times does not work?
What number would work for other users?
Is there something we can detect that causes it to fail?  Do we need to
wait for some specific amount of time?
--
ALL:   A witch!  A witch!
WITCH: It's a fair cop.
ALL:   Burn her!  Burn her!  Let's make her into a ladder.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

I'm suspecting a timeout, since as far as I know it only happens with a broken connection between a client and the X server.  I'll see what more I can find out.

I've got a bit more information.   It appears that XtOpenDisplay() is unable to obtain an authorized connection to the X server for some period of time following a connection break.   The following list was taken using vim with the most recent patch I have, and setting xterm_dpy_retry_count=10.  Each entry gives the result of a single XtOpenDisplay() attempt to connect to the server.

Two quick additional notes:

the example program should be saved as utl1.c
if you use a smaller wait between retries (ex using usleep() instead of sleep()...)  many more retries are required.

I've got a bit more information.   It appears that XtOpenDisplay() is
unable to obtain an authorized connection to the X server for some
period of time following a connection break.   The following list was
taken using vim with the most recent patch I have, and setting
xterm_dpy_retry_count.  Each entry gives the result of a single
XtOpenDisplay() attempt to connect to the server.

Thanks for taking the time to look into this.
It appears it's not so much the number of tries, but some time until it
works.  So perhaps we can restrict the time for retries to perhaps
something linke 3 seconds?  That would also avoid getting stuck when
each try is slow.  So we would do something like:
- try at least once
- Stop trying when over the time limit (e.g. 3 seconds)
- Stop trying after some count (e.g. 50).
So with your attempts, if you add a time stamp and try a few times, we
should be able to see if there is some number of seconds after which it
works again.
--
I once paid $12 to peer at the box that held King Tutankhamen's little
bandage-covered midget corpse at the De Young Museum in San Francisco.  I
remember thinking how pleased he'd be about the way things turned out in his
afterlife.
(Scott Adams - The Dilbert principle)
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

I will try this with vim.  From working with utl1.c, it looks like it's about 8 to 10 seconds here.  I see what it turns out to be with vim.

I have prepared a new patch to vim-8.1 with the changes you have suggested.  In testing this, I find that the individual XtOpenDisplay() connection attempts can vary greatly in the time required.  If XtOpenDisplay() encounters a significant number of "No protocol specified" errors, (apparently from XDisplay() which it calls), the time required for an individual XtOpenDisplay() connection attempt can be over 30 seconds.  I have not seen this happen more than once per X server connection break, though it usually happens once per connection break.
Accordingly, I have set the following in the patch:
#define XTERM_DPY_RETRY_TME_LIM 60
#define XTERM_DPY_RETRY_CNT_LIM 50

Here is the latest patch to vim-8.1:
diff -ru vim81/src/os_unix.c vim81.new/src/os_unix.c
--- vim81/src/os_unix.c	2018-05-12 11:42:33.000000000 -0400
+++ vim81.new/src/os_unix.c	2018-12-10 10:19:27.416400966 -0500
@@ -1686,13 +1686,18 @@
  */
 static int x_IOerror_handler(Display *dpy);
 static void may_restore_clipboard(void);
-static int xterm_dpy_was_reset = FALSE;
+static int xterm_dpy_retry_count = 0;
+static time_t xterm_dpy_retry_start_time;
+
+#define XTERM_DPY_RETRY_TME_LIM 60
+#define XTERM_DPY_RETRY_CNT_LIM 50

     static int
 x_IOerror_handler(Display *dpy UNUSED)
 {
     xterm_dpy = NULL;
-    xterm_dpy_was_reset = TRUE;
+    xterm_dpy_retry_count = XTERM_DPY_RETRY_CNT_LIM;
+    xterm_dpy_retry_start_time=time(NULL);
     x11_window = 0;
     x11_display = NULL;
     xterm_Shell = (Widget)0;
@@ -1712,22 +1717,30 @@
     static void
 may_restore_clipboard(void)
 {
-    if (xterm_dpy_was_reset)
+    time_t elapsed;
+
+    if(xterm_dpy_retry_count > 0)
     {
-	xterm_dpy_was_reset = FALSE;
+	elapsed=time(NULL)-xterm_dpy_retry_start_time;
+	if(elapsed < XTERM_DPY_RETRY_TME_LIM)
+	{
+	    --xterm_dpy_retry_count;

 # ifndef LESSTIF_VERSION
-	/* This has been reported to avoid Vim getting stuck. */
-	if (app_context != (XtAppContext)NULL)
-	{
-	    XtDestroyApplicationContext(app_context);
-	    app_context = (XtAppContext)NULL;
-	    x11_display = NULL; /* freed by XtDestroyApplicationContext() */
-	}
+	    /* This has been reported to avoid Vim getting stuck. */
+	    if (app_context != (XtAppContext)NULL)
+	    {
+		XtDestroyApplicationContext(app_context);
+		app_context = (XtAppContext)NULL;
+		x11_display = NULL; /* freed by XtDestroyApplicationContext() */
+	    }
 # endif

-	setup_term_clip();
-	get_x11_title(FALSE);
+	    setup_term_clip();
+	    get_x11_title(FALSE);
+	}
+	else
+	    xterm_dpy_retry_count = 0;
     }
 }
 #endif
@@ -7468,6 +7481,9 @@
 	{
 	    xterm_dpy = XtOpenDisplay(app_context, xterm_display,
 		    "vim_xterm", "Vim_xterm", NULL, 0, &z, &strp);
+	    if (xterm_dpy != NULL)
+		xterm_dpy_retry_count = 0;
+
 #if defined(HAVE_SETJMP_H)
 	    mch_endjmp();
 #endif

I should further note that I have not seen it require more than 10 retries to re-establish the X server connection.  The individual attempts are usually very quick initially, but eventually a "No protocol specified" condition occurs and involves a longer duration attempt.  It generally succeeds in re-establishing the connection shortly afterwards.

As "No protocol specified" error sometimes indicates that ~/.Xauthority is no longer valid for the current X session, it appears to me that the first thing we should do after the X server restarts is to correct/update it using xauth(1) in accordance with the new instance of the X server which was restarted, so that new connections will be granted with the very first attempt at establishing connection.  If so, then with that corrected/updated ~/.Xauthority,  we could be free from a shaky combination of retry and timeout.

As far as I know, ~/.Xauthority is updated with the correct information at login by programs that start the X server such as xdm or startx.  The XOpenDisplay() call done by XtOpenDisplay() as called from a client causes ~/.Xauthority to be read to obtain the requisite authorization for the particular display to be opened.  In checking the connection dialog between the client and the X server, the actual connection request generated by the XOpenDisplay() call appears to be incorrect for some period of time.  It appears as though the authorization protocol identification and data is omitted during that period.  There were a few other issues as well, such as unix socket file entries missing or otherwise unavailable.

nuko8's comment led me on to something that seems important.
In the cases I am testing, I am running vim from a shell in a GNU screen instance.
What is happening with the retries is that the X server connection infrastructure and authentication data is being rebuilt by the new instance of the X server.  This seems to take somewhat more than 30 seconds once the new server instance starts up.  When vim attempts to reconnect initially, the X server connection infrastructure and authentication data is either obsolete, absent altogether, or incomplete.
A remote vim session accessed via ssh with X forwarding appears to experience similar circumstances.
If a timeout is used as part of our patches as they are now, I am thinking that a connection would need to be re-established before the timeout expires, or the X11 cut/paste will remain disabled until the connection is rebroken and re-established before the timeout expires.

Testing with "top" on a machine with 24 vim processes active while disconnected from an X server shows little or no cpu usage by them.

The unfinished fix was accidentally included in patch 8.1.0615 (big patch was touching the same file).
I assume this won't completely fix the problem, as the retry count is low.  But it might help in some situations.

closing this for now. Please let us know if the "fix" is working or not.


ESMValTool Docker container

Did we want to consider developing an automated build of an ESMValTool docker image on Docker Hub, triggered by PR changes to a github controlled Dockerfile (centos:6 base image ?) with all the environment dependencies containerized?

Closed by https://github.com/ESMValGroup/ESMValTool/pull/26 and the awesomeness of @mkuzak 😄


added free java Rest Api and SpringBoot courses

courses are easily available for free for the beginners on youtube.
What does this PR do?
Add Resource(s) | Remove Resource(s) | Add info | Improve Repo
For resources
Description
Why is this valuable (or not)
How do we know it's really free?
For book lists, is it a book?
Checklist:

[ ] Not a duplicate
[ ] Included author(s) if appropriate
[ ] Lists are in alphabetical order
[ ] Needed indications added (PDF, access notes, under construction)

what's '.vs'?


Don't open transcript/protein accordion by default if user closed all

When there is nothing expanded, added a dummy value to the expandedIds array so that its not empty which was causing the first one to be always open
Type

[ ] Bug fix
[ ] New feature
[x] Improvement to existing feature

Related JIRA Issue(s)
https://www.ebi.ac.uk/panda/jira/browse/ENSWBSITES-1351
Importance
N/A
Description
As per jira ticket, an improvement to the behavior of the expanded transcript accordion when the user closed all accordion, it should all be closed in the protein tab as well and vice versa.
Deployment URL
http://transcript-accordion.review.ensembl.org
Views affected
Entity Viewer
Other effects
N/A

[ ] I have checked any requirements for Google Analytics
[ ] I have created any required tests
[ ] The PR has as its final commit a WASM/JS update commit if any Rust files were updated.

Possible complications
N/A

If I follow these steps I still see the transcript accordion opened:

Close all transcript accordions
Go to Gene function
Click on a protein accordion
Head back to Transcripts and see the accordion of the selected protein's transcript opened

I've also attached a screen recording of this below.
https://user-images.githubusercontent.com/32512909/145229481-ca0eb054-6a97-4615-a2f5-4e06893094be.mov

If I follow these steps I still see the transcript accordion opened:

Close all transcript accordions
Go to Gene function
Click on a protein accordion
Head back to Transcripts and see the accordion of the selected protein's transcript opened

I've also attached a screen recording of this below.
Screen.Recording.2021-12-08.at.14.51.06.mov

Hi @imransl , thats the right behavior. As soon as you interact with the accordion in either view (Transcript list or protein view) the same view is reflected in the other view. So for example if you have no accordion open in the transcript view and then head to protein view then you wont see any accordion open.  And if you head back to the transcript view you will still see no accordion open.
But now if you interact with the view on protein and decide to open one or two accordion then when you go back to the transcript view the same transcript accordion will be open.
Hope that makes sense.

@ens-ma7 Yes that makes sense. Thanks for the explanation.

Interesting and clever, but too hacky. It injects a fake transcript id into the list of transcript ids and relies on the knowledge that no real transcript will have an id that matches this fake id. For anyone reading this code, the implementation of this behaviour will be a surprise, while it is generally advisable to have as few surprises in the codebase as possible.

Interesting and clever, but too hacky. It injects a fake transcript id into the list of transcript ids and relies on the knowledge that no real transcript will have an id that matches this fake id. For anyone reading this code, the implementation of this behaviour will be a surprise, while it is generally advisable to have as few surprises in the codebase as possible.

There wont be a transcriptID whose name is called noselection :) I could add a comment there to explain for others to understand :)  Otherwise what solution will you suggest?


ci: clean up workflow files

This pull request cleans up workflow files:

I made the task names consistent in casing
I removed two seemingly unused workflows
I fixed coding-conventions.yml, which was corrupted in a rebase
I removed the updating of CHANGELOG.md in the GitHub release workflow, as that doesn't work with branch protections
I took the opportunity to update package.json scripts

@innocenzi looks like there are still a few cases of the brand names being off (another phpstan, etc.). Once that's cleaned up, I like it and let's get this merged.
Side note, this is one of those things we should probably document in contributing.

@aidan-casey Thanks! Here I was, thinking I was clever with my perfectionism, and I missed one 😔


Highlight all occurrences of selected word

Added a configurable option that highlights all occurrences of the selected word within the current file in the text editor.

@akien-mga Code looks good.

Looks fine indeed, nice feature :)

Very nice feature but this behaviour is expected?

It seems to look good to me, highlight would be for word matches while underlined would be for substring matches in bigger word (though I would expect dock used as an argument on lines 11 and 16 to be highlighted too).

@akien-mga weird thing is that I only see one rect being added in the diff and it highlights all the word: https://github.com/godotengine/godot/pull/4076/commits/b0488cacf27cb98c4a0dd58661c83262236e18a0#diff-71709879dd36a28d4ff43c570b3b4c8bR823

That's right :D Quite peculiar hehe.

Oh now I understand what @lorenzobeccaro meant. He underlined himself the words that should have been in his opinion highlighted, but are not.

Yeah, it's not meant to highlight the substrings, but I don't see why it's not highlighting the arguments, i'll take a look into it.

I noticed this bug only happens if there is one or more substring in the same line before the word that fails to be highlighted:
dock # dock is selected
fun _docks(dock) # fails to highlight the parameter dock
fun _hello_there(dock)_docks # successfully highlights the parameter dock

I also noticed that it will only highlight the occurrences when selecting with double click. Selecting with mouse drag or shift won't work. Is that the intended behaviour?

The cause seems to be that findn won't be called twice if the word found in the line is a substring, when calling _get_column_pos_of_word before iterating through the line columns.

I noticed that too, have to add some kind of loop to check until we get no matches. Just checked how sublime text handles mouse drag or shift, seem like it should be added.  Just fixed it here

I'll try to fix the other issue then i'll send in a pull request.

I guess the other way would be to check the line for check the line for the word every loop, though that seems wasteful.

Try this. I can't test it right now, but it should loop until it finds the correct word or until it reaches the end of the line:

int TextEdit::_get_column_pos_of_word(const String &p_key, const String &p_search, int p_from_column) {
	int col = -1;

	if (p_key.length() > 0 && p_search.length() > 0) {
		if (p_from_column < 0 || p_from_column > p_search.length()) {
			p_from_column = 0;
		}

		while (col==-1 && p_from_column<=p_search.length()) {
			// match case
			col = p_search.findn(p_key, p_from_column);

			// whole words only
			if (col != -1) {
				p_from_column=col;

				if (col > 0 && _is_text_char(p_search[col-1])) {
					col = -1;
				} else if (_is_text_char(p_search[col+p_key.length()])) {
					col = -1;
				}
			}

			p_from_column+=1;
		}
	}
	return col;
}

That seems to have done the trick. Thanks :) i'll send in the pull request.


Apply containerd.io/snapshot.ref label to layers during pull

Current Behaviour
Running the command ctr image pull ... --snapshotter=xxxx will pull an image but not apply the containerd.io/snapshot.ref label to the layers such that the snapshotter cannot return ErrAlrExists when Prepare() is invoked.
Proposed Behaviour
Apply the label so that when the Prepare() function of the snapshotter is invoked it can check the cache and return ErrAlrExists if it is present, if not then processing happens as normal.
Rationale
We have a custom snapshotter that provides a caching function so that we can have a "here's a snapshot we prepared earlier" facility. We have some clients that are able to pull/unpack/run that use this facility but when we try to use the ctr command it doesn't go to plan. (We also have our own content store plugin that will also check a cache - this works fine with any pull mechanism.)
Clients that use the client.Pull() API with the Unpack option will add the containerd.io/snapshot.ref label that the snapshotter can detect and be allowed to return ErrAlrExists. Similarly, the PullImage() API will apply labels. However, when we use ctr, which uses the image.Unpack() API, no labels are passed to the Prepare() function of our snapshotter so it cannot check our cache and tell containerd that the snapshot already exists (containerd will not accept ErrAlrExists being returned by the snapshotter if that label is not present).
Both k8s and k3s can use the PullImage() API so will work with our snapshotter.
Note, this draft does not respect the disable_snapshot_annotations configuration setting like the PullImage() API (neither does client.Pull()). This draft PR has been opened to examine these issues.
Further references: https://github.com/containerd/containerd/discussions/8942 and https://github.com/containerd/containerd/discussions/8977
Files affected:

rootfs/apply.go

Add label to layer

Sorry for the slow reply. Overall looks good about the logic. Could you fix the code for the linter errors?
L133
File is not `gofmt`-ed with `-s` (gofmt)

L156
SA9003: empty branch (staticcheck)

@dmcgowan I'd like to pursue this PR as I think it's needed for proper operation of snapshotter plugins. I will attempt to forward fit this to 2.0 if that would help.


manuskript: 0.8.0 -> 0.9.0

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/manuskript/versions.
meta.description for manuskript is: '"A open-source tool for writers"'.
Release on GitHub
Compare changes on GitHub

Checks done (click to expand)

built on NixOS
/bin/sh passed the binary check.
1 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 0.9.0 with grep in /nix/store/5bv7wk7p2b854kglgaq6yqp6gkya3lpz-manuskript-0.9.0
directory tree listing: https://gist.github.com/12eab7e7253ad014432c0e7a040c4b9d
du listing: https://gist.github.com/3e1a39d01ee043fc09984673d0041334

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)
1 package rebuild(s)
1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
manuskript

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5bv7wk7p2b854kglgaq6yqp6gkya3lpz-manuskript-0.9.0 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A manuskript https://github.com/r-ryantm/nixpkgs/archive/9dc79bef92ed1848dda04efa6d87a66fc8948b6e.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5bv7wk7p2b854kglgaq6yqp6gkya3lpz-manuskript-0.9.0
ls -la /nix/store/5bv7wk7p2b854kglgaq6yqp6gkya3lpz-manuskript-0.9.0/bin

cc @steveej for testing.

@GrahamcOfBorg eval

Built and tested locally :+1:


License Clarification - AGPL or CC BY-SA 3.0

Thanks for sharing this theme, much appreciated. I'm just looking for a clarification on the license as it appears to differ between the Hugo Themes gallery and the LICENSE in Github.
The Hugo themes gallery is showing a license of Creative Commons Attribution 3.0 Unported and the LICENSE file is AGPL.
Not a lawyer, but with my understanding of the AGPL would mean that any hugo sites using this theme would also need to be licensed under the AGPL, is that your intent? Edit: I think this may be only if the theme is modified.
Thanks for any clarification on this.

Thank you for bringing this to my attention! There was some confusion on my part after creating the theme when I was still figuring out the licensing.
I've updated the license to Apache License 2.0, which is more permissive and in line with my other Hugo theme, Introduction.


support for imageUrl in invitation

This can potentially provides a default icon for credentials issued from a connection. Or show a logo for the verifier on a proof request?
for AIP1:
{
"@type": "did:sov:BzCbsNYhMrjHiqZDTUASHg;spec/connections/1.0/invitation",
"@id": "d50a86d1-4a0e-4261-b96e-d95f84a7dac3",
"recipientKeys": ["7UsJMdBHBGKR5cZAF3WCZvgvPKufX4gq7c6RnsAwf37m"],
"label": "faber.agent",
"imageUrl": "https://robots.txt/ian",
"serviceEndpoint": "http://host.docker.internal:8020"
}

see imageUrl
for AIP2:
???

I know this is supported in some wallets. If you run through the BC Wallet demo with Lissi it will display the logo. I could probably get this working quickly, the question is if we want to pursue this since it will probably be replaced by OCA styling in the future

resolved by: https://github.com/hyperledger/aries-mobile-agent-react-native/pull/648

@wadeking98 How is this testable with BC Wallet at this time?

@wadeking98 How is this testable with BC Wallet at this time?


Pagination support for Expand API

Add support for pagination into the Expand API
Should ideally make use of a generic pagination system over the dispatcher that can be shared with Lookup

What is the current status of this issue?
It is highly required by our processes. Can we make a contribution to this or is there any ongoing development for this?

@mstrYoda No updates and not currently scheduled. We would be happy to accept a contribution.

One thing to note: we should support continuation cursors, not pagination

One thing to note: we should support continuation cursors, not pagination

Thank you for your response, I will investigate on this for a while 👍

@mstrYoda Cursor support has landed for LookupResources and ReadRelationships. if you still intend to do this, you can reuse the cursor package.

@mstrYoda Cursor support has landed for LookupResources and ReadRelationships. if you still intend to do this, you can reuse the cursor package.

Thank you for all of your effort, we are closely following new releases of spicedb the cursor support made us very happy :)
If I wait for a while I guess this will be implemented :D I will take a look at the new developments 👍

@mstrYoda Given that cursors will land soon-ish for LookupSubjects, I suspect we won't add cursors to Expand unless there is a need. Do you still need this support?

@mstrYoda Given that cursors will land soon-ish for LookupSubjects, I suspect we won't add cursors to Expand unless there is a need. Do you still need this support?

Hi @josephschorr, cursor for LookupSubjects is cover most of our requirements. Thank you, I do not see any urgent need for Expand.


Bump github-pages-deploy-action to v4.1.5

Fix security issues in javascript
Fix bug when some options are combined

Superseded by #92


Add email notification when there are shared users.

Send an email notification via MailJet whenever shared users are added to an expense.

#6

The Xml files which are showing up in the diff are configuration files which are no longer applicable to the project. The keys are currently checked in until we find a better solution to manage the keys. Will take this up separately.


Add custom page-load time tracking system

Measures performance from initial network request through until all images on the page are loaded. Sends events to Snowplow so we can aggregate the data and improve perf on key areas. You can use the two debug constants at the top of ViewLoad to console log or noty the info to check it against reality.

Coverage decreased (-0.01%) to 56.632% when pulling 0475cb3f30c53a88adcbd6bd6e1280dfe9c108d5 on track-perf into 643a2ca49cda97ce1d9ea790ca0bfaba7d24f589 on master.

Coverage decreased (-0.01%) to 56.632% when pulling 682a7468a2bbdd93bb4b93cfdfa271c06eb3f9eb on track-perf into 643a2ca49cda97ce1d9ea790ca0bfaba7d24f589 on master.

Coverage increased (+0.01%) to 56.646% when pulling 663e90fca071c991efab35c1665c52eb22b51b8a on track-perf into 30d1a8ada4eb7d06c6aa3b59a7774f633ae8e61d on master.

Coverage remained the same at 56.632% when pulling 6fbd9254f2b51732650a994fa5ce6acbc4ebd2b3 on track-perf into 5112eac696fe366ba962726649a6bcaf44dc3977 on master.

Coverage increased (+0.1%) to 56.773% when pulling 6fbd9254f2b51732650a994fa5ce6acbc4ebd2b3 on track-perf into 5112eac696fe366ba962726649a6bcaf44dc3977 on master.


adding a script for tool access

@MikeEdgar for usage with the instance profiler it seems handy to automate the 1/2 steps for tool access.  If it seems ok, I can further refine it and the README.
Also would there be any objection to adding bootstrap.servers=<bootstrap_server_host> into the app-services.properties?

@shawkins this is good, thanks. I think adding the bootstrap server to the properties would be fine.

lgtm


build: generate and install pkg-config files on mingw

Mingw-w64 is typically used in conjunction with a unix-like environment,
whether as a cross compiler on linux or as a 'cross' compiler with
cygwin (still win32 and win32, but kinda like cross-compiling), and the
use of pkg-config files within this sort of setup is rather common.
Signed-off-by: Marty E. Plummer<EMAIL_ADDRESS>

To achieve the desired effect I added a new variable, SFML_OS_WINDOWS_MINGW to
specify that this is a windows compile using the mingw-w64 runtime and libraries. I could
just add a MINGW check to the if(SFML_OS_LINUX OR SFML_OS_FREEBSD) check,
which would reduce the patch to a single addition and subtraction, but I was attempting to
match your current conventions of wrapping ''global'' variables into SFML ''wrapped'' ones.

Alternatively one can just generate pkg-config files for all platforms except msvc,
as just about every other platform there is can consume pkg-config files, being
as they're all POSIX to some degree or another.

Why close it?

@eXpl0it3r because there apparently is already a commit in master which does this, more
or less. its 5fe5e5d6d7792e37685a437551ffa8ed5161fcc1, and since gentoo unconditionally
sets SFML_INSTALL_PKGCONFIG_FILES in its ebuild its not needed on 2.4.2 if we add this
commit as a patch.

I see, thanks for the clarification and the contributions! 🙂


conmon fuzz target

I ran into https://github.com/containers/conmon/issues/315#issuecomment-1296141621 the other day and then I found https://ostif.org/wp-content/uploads/2022/06/CRI-O-audit-by-ada-logics-chainguard-ostif.pdf where scenarios like that were included in the threat model

... and
input from the container’s are also handled in Conmon. These are areas of potential attack
surface against Conmon.

and as far as I understand there should be a fuzz target:

We also developed a fuzzer for Conmon to analyse the logging and parsing routines in conmon/src/ctr_logging.c.

@DavidKorczynski I can't seem to find that fuzz target anywhere. Is there any chance you could point me in the right direction?

This particular fuzz target was not uploaded to OSS-Fuzz (or anywhere else public) -- it was a bit non-timeless and a bit forced into the build system so we decided to leave it out.

Quite interesting with https://github.com/containers/conmon/issues/315#issuecomment-1296141621
I'll try to dig out the fuzz target and see why we did not catch that.

it was a bit non-timeless and a bit forced into the build system so we decided to leave it out.

Got it. Thanks!

I'll try to dig out the fuzz target and see why we did not catch that.

My guess would be that it wasn't linked against libsystemd (where messages and their lengths are used by sd_journal_sendv).

FWIW as far as I understand conmon is being rewritten in Rust so I'm not sure whether it would make sense to resurrect that particular fuzz target long term. It would certainly help to test patches and make sure new regressions aren't introduced though.

My guess would be that it wasn't linked against libsystemd

Before I forget I think https://google.github.io/oss-fuzz/getting-started/new-project-guide/#static-and-dynamic-linking-of-libraries makes it unnecessarily hard to fuzz codebases with external dependencies like glibc and libsystemd. Because of that I missed (and keep missing) quite a few bugs and have to run a bunch of fuzz targets outside of OSS-Fuzz. It would be great if it was possible to lift those restrictions on at least ClusterFuzzLite. As far as I can remember I already kind of complained about that (and it didn't work out at the time) but maybe it's time to try again.
Anyway I'd keep this issue open. Rust should eventually help to eliminate some bugs but it would still make sense to fuzz the parser to catch panics, stack overflows and stuff like that.


"sudo make install" returns "make: Nothing to be done for 'install'."

The last step won't work, guessing because there is no install file in the directory? Using Ubuntu (POP!_OS) 19.04
Everything else went sooth. Can open the app just fine, but it does not install to the system.

@DaveFlowberries @carlosnewmusic

Be sure to have installed all the required dependencies

sudo apt-get install build-essential qtchooser qt5-default libjansson-dev libcurl4-openssl-dev git zlib1g-dev automake autoconf libtool libjansson-dev

make sure to have c-vtapi installed first
run the commands as follows:

qmake
make
sudo make install

You can run qtchooser -run-tool=qmake -qt=5 to explicitly choose qt5's qmake tool.

Facing the same issue with Ubuntu 20.04 LTS. Every dependence was installed. But same thing happening with sudo make install
$ sudo make install
make: Nothing to be done for 'install'.

@DaveFlowberries @carlosnewmusic

Be sure to have installed all the required dependencies

sudo apt-get install build-essential qtchooser qt5-default libjansson-dev libcurl4-openssl-dev git zlib1g-dev automake autoconf libtool libjansson-dev

make sure to have c-vtapi installed first
run the commands as follows:

qmake
make
sudo make install
You can run qtchooser -run-tool=qmake -qt=5 to explicitly choose qt5's qmake tool.

Not working,

Executable file is located in ~/qt-virustotal-uploader/VirusTotalUploader
You can copy it anywhere or to the PATH (~/bin)
It also makes sense to strip (bin and lib) to reduce size:
strip --strip-unneeded
For the future: sort files by type in the file manager and look for executable (binary) files.


🚀 [Category] - Portfolio

Category Request
Developer Portfolio
Purpose
A user can take inspiration from the curated list of developer portfolios
Resource Examples
Examples

https://leerob.io/

Code of Conduct

[x] I follow Contributing Guidelines of this project.

Like this comment, if you want to add this category.

completed 🔥


Design backend solution

to support:

new web app, e.g. viz tool, and the data behind
current flagging web app, eventually new flagging app
twitter bot
other?

brainstorming: https://docs.google.com/document/d/1kark84Ocmmx6d7lX0O6OEswo2yWbQJ1uBVAyWDDW83U/edit
current architecture: https://docs.google.com/presentation/d/1nppcS1IdsHbY6OSaPhRoWEn9lanqvbDgCsftiv4pmq4/edit#slide=id.g83e65ca87f_0_103


ABI issue with json object between 2 shared libraries

Bug Report

What is the issue you have?

Crash somtimes, not always

Please describe the steps to reproduce the issue. Can you provide a small but working code example?

main application uses 1 shared libraries a.so that uses another library b.so.
"json" object J is instanciated by a.so
J is passed by const reference to b.so

What is the expected behavior?

passed json object J can be used throughout b.so

And what is the actual behavior instead?

Sometimes it works, sometimes it crashes, ex:
0x00007ffff7b3bd24 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>::basic_json(nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> const&) ()
I tried to pass the json object by copy instead but eventually it will crash elsewhere down the road.

Which compiler and operating system are you using? Is it a supported compiler?

gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2)

Did you use a released version of the library or the version from the develop branch?

develop branch

If you experience a compilation error: can you compile and run the unit tests?

Feature Request

Describe the feature in as much detail as possible.

Right now the only way to work around the issue is to remove json from the shared library interface and just serialize/deserialize with string.

Include sample usage where appropriate.

I have the same issue with master branch:
0x00007ffff7b3b034 in nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>::basic_json(nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer> const&) ()

It seems that adding a simple print work around this issue:
bool setOptions(const json &options) override {
cout << options << endl;
....
}


Ridiculous "This exists for static type-checking purposes only" error message

Version 2.1.23.
System.InvalidOperationException: This exists for static type-checking purposes only.
  at ExtendedXmlSerializer.ContentModel.SerializerMarker.ExtendedXmlSerializer.ContentModel.IWriter<System.Object>.Write (ExtendedXmlSerializer.ContentModel.Format.IFormatWriter writer, System.Object instance) [0x00000] in C:\projects\extendedxmlserializer\src\ExtendedXmlSerializer\ContentModel\SerializerAdapter.cs:77
  at ExtendedXmlSerializer.Write`1[T].Execute (ExtendedXmlSerializer.ContentModel.Format.Writing`1[T] parameter) [0x00000] in C:\projects\extendedxmlserializer\src\ExtendedXmlSerializer\Write.cs:43
  at ExtendedXmlSerializer.Serializer`2[TRead,TWrite].Serialize (TWrite writer, System.Object instance) [0x00000] in C:\projects\extendedxmlserializer\src\ExtendedXmlSerializer\Serializer.cs:41
  at ExtendedXmlSerializer.ExtensionModel.Xml.ExtendedXmlSerializer.Serialize (System.Xml.XmlWriter writer, System.Object instance) [0x00000] in C:\projects\extendedxmlserializer\src\ExtendedXmlSerializer\ExtensionModel\Xml\ExtendedXmlSerializer.cs:40

Code that causes the issue:
var m = new XMLModMetadata();
m.Author = "me";
m.Name = "test";
m.Description = "desc";
var ms = new MemoryStream();
using (var xw = XmlWriter.Create(ms)) {
    new ConfigurationContainer().Create().Serialize(xw, m);
    xw.Flush();
}

Hey @zatherz thanks for writing in.  I agree... this error is SO ridiculous that I can't even reproduce it. 😆  I've created a passing test with your provided code here:
https://github.com/wojtpl2/ExtendedXmlSerializer/blob/8fd479a1045cc9191ff3480024913f1fd0d6fbdc/test/ExtendedXmlSerializer.Tests/ReportedIssues/Issue266Tests.cs#L10-L21
Perhaps this might be a caching issue?  Try deleting your C:\Users\<user>\.nuget\ExtendedXmlSerializer folder and do a rebuild of your solution. 🤞

Since we have not heard a reply, closing this issue for now.  Please feel free to comment if any questions remain.


Refactor HTML5 CSS configuration

Add tests for HTML5 CSS configuration and refactor code. The PR should not change the expected output.

Do the refactored checks for absolute paths also address the bug reported in #3016?

Could a similar approach be applied to header/footer files and other cases where various toolkit parameters currently use different approaches to path resolution?
Aligning these behaviors would allow us to resolve #2815 and perhaps a few related issues.

@robander No, this is just refactoring the code.
@infotexture Yes, but not in this PR.


Multi-object table display

Jockey should be able to display combinations of Juju objects in a table when prompted by the user.  For example, a user may be using Jockey to filter a subset of units, but may want to see both unit names and their corresponding IPs and whether they are principal/leaders or not.
Sub-tasks:

Decide what CLI flag is best suitable for this functionality.  One option might look like jockey unit ... -o unit,ip
Add table display functionality, where columns are aligned and an arbitrary number of rows can be requested

How is something like this?

That's pretty close to what I had in mind.  However, I want it to be easy to only show certain fields.  I see how this is working with dot notation, but my original conception was something like this:
juju-jockey -f tests/samples/k8s-core-juju-status.json u -o i,h,a
+----------+-----------------+
| <unit 1> | <ip-address 1>  |
|          | <hostname 1>    |
|          | <application 1> |
+----------+-----------------+
| <unit 2> | <ip-address 2>  |
|          | <hostname 2>    |
|          | <application 2> |
+----------+-----------------+
| <unit 3> | <ip-address 3>  |
|          | <hostname 3>    |
|          | <application 3> |
+----------+-----------------+

-o is completely arbitrary and it can be replaced with something else.  If there is a more elegant way to represent this idea in the CLI, I am open to it.
I envisioned this as a human-readable output that basically won't ever be redirected to anything else.
If you think there may be a simply better way to implement this flavor of functionality, do let me know.

Oh! I understand now.
Output multiple user-selected fields (instead of or in addition to the ability to do all).
I have been baking up a change that adds output formatting classes; e.g., you could do -O json for valid JSON output or -O yaml for YAML, etc. etc.
So, essentially this paired with that would give numerous options to manipulate the output of objects.

Interesting idea.  Let's explore this more.
What are the details on the contents of -o json and -o yaml outputs?  Is that snippets of juju status --format json, or is it a simple mapping analogous to the table I posted above?  I bring this up because my table idea incorporates both a specified output format and some related objects (i,h,a).  Setting the output to json or yaml format does not require those related objects.  Perhaps there is a way to marry these ideas.
Just spit-balling from here on out:
I am choosing a plausible command as a starting point for the discussion: juju-jockey -f tests/samples/k8s-core-juju-status.json u -o table i,h,a.  Note that I added table, compared my prior version of this command.  I would expect this to return the table I shared above.  So that begs the question: what would juju-jockey -f tests/samples/k8s-core-juju-status.json u -o json i,h,a and juju-jockey -f tests/samples/k8s-core-juju-status.json u -o yaml i,h,a return?
Even if the answer is not obvious, I am biased in favor of feature compatibility (perhaps not parity) here.  That is, if -o table <list of juju object types> is a thing, then it should also be a thing for -o yaml and -o json, even if that list of objects is used differently in each case.

Another option may be to introduce new syntax to change how <object> is parsed (relevant README.md line).
Consider the following command:
juju-jockey u,m,i a~nova-compute
That u,m,i may be read as "get a mapping of units to their machines and ip-addresses."  Putting it all together, this would return a mapping of each nova-compute unit to its machine number and IP address.
Simply defaulting to a table format in these cases may be the solution here; it would remove the need for -o table and eliminate the heterogeneity with <list of juju object types>.
To explore how this syntax would interact with the other -o values, I need to return to my initial question: what will -o json and -o yaml return in a normal context?  Beyond that, what would be the most useful behavior in each case?
I could see us dumping extra information into the json and yaml outputs, such as model, controller, datetime, executed query, etc.  None of that information makes sense in the table output, but would essentially be free in the other formats.
Summary:
juju-jockey u a~nova-compute                # Normal list of names

juju-jockey u,m,i a~nova-compute            # Table format

juju-jockey u a~nova-compute -o yaml        # ?

juju-jockey u,m,i a~nova-compute -o yaml    # YAML with unit->machine,IP mappings plus extra info

juju-jockey u a~nova-compute -o json        # ?

juju-jockey u,m,i a~nova-compute -o json    # JSON with unit->machine,IP mappings plus extra info

Maybe I am way over-thinking it.  Please do scrutinize.

The object method is more intuitive and feasible in the code since it can traverse the virtual fields. We would need to expand those virtual fields to accommodate the object shorthand.
It reads like this:

I want to analyze units, filtering these fields to these values, outputting the unit fields @machine and @application

This provides the added benefit of reaching into those related objects for their fields.

We hit a wrinkle with the @ notation for the virtual and existing fields in Juju. I recall there being a machine field on Units. Placing a virtual field to get the related Machine object on machine would shadow the value there. This is not necessarily bad since that value can now be retrieved via @machine.name. It's a matter of your personal preference.
It may be more natural to drop the @ where it relates to an object relationship. I feel @ is appropriate where the actual value doesn't necessarily exist anywhere in the status but as something we calculated.

As a result of my interpretation, the final implementation would look like this:
juju-jockey unit,unit.machine,unit.ip

Agreed. We should shadow those properties as those values are obtainable through other means. It should just be documented.
--
John Lettman
@.***
On Sat, Sep 21, 2024, at 2:27 AM, Lewis C Chamberlain wrote:

I've been thinking about this for a while. I think we should just remove the @.
I cannot image me doing anything other than adding the @ in all of my queries all the time. Why would I ever use unit.machine (which currently fails in some cases) when @.***` works every time. There does not seem to be a good reason to waste a key stroke.
—
Reply to this email directly, view it on GitHub https://github.com/LCVcode/jockey/issues/4#issuecomment-2365025212, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5BG7S2MQQ45FPOBF5HIELZXUGUBAVCNFSM6AAAAABNVKN2TCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRVGAZDKMRRGI.
You are receiving this because you commented.Message ID: @.***>

I feel similarly about the command you proposed:
juju-jockey unit,unit.machine,unit.ip

It is not clear to me how this is advantageous compared to:
juju-jockey unit,machine,ip

I wouldn't necessarily say those are advantageous rather than easier to implement and consistent into the field interface (e.g. how the field is extracted). Alternatively, though, it simply can be "implied" that were just working over the same object and implicitly pass those fields with the same object.
I'm not sure that entirely makes sense as I explain it, but tldr is that it's possible too.
--
John Lettman
@.***
On Sat, Sep 21, 2024, at 2:29 AM, Lewis C Chamberlain wrote:

I feel similarly about the command you proposed:
juju-jockey unit,unit.machine,unit.ip
It is not clear to me how this is advantageous compared to:
juju-jockey unit,machine,ip
—
Reply to this email directly, view it on GitHub https://github.com/LCVcode/jockey/issues/4#issuecomment-2365025891, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5BG7S6CTZR6ZLVWLHIY2LZXUG43AVCNFSM6AAAAABNVKN2TCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRVGAZDKOBZGE.
You are receiving this because you commented.Message ID: @.***>

I think having it be implied makes the most sense.  This would make it easier and faster to use in the wild.  This feature has a narrow use case, but we should prioritize convenience of use over ease on implementation, in my opinion.


Updated the claims code

made the code shorter, removed the output component, which was making the notebook so huge
Relevant notebook can be viewed at: https://github.com/97harsh/task-vt/blob/claim_extraction/contradictory_claims/notebooks/2T.4.5 Claim extraction.ipynb

Is there any chance you can add some of the code in these notebooks as python modules that are imported in the notebooks? This has several benefits:

allows for code quality checks
Allows for code review in this PR
Makes notebooks more reproducible
Removes clutter from notebook so the notebooks can be focused on the scientific story rather than the code that accomplishes it

Is there any chance you can add some of the code in these notebooks as python modules that are imported in the notebooks? This has several benefits:

allows for code quality checks
Allows for code review in this PR
Makes notebooks more reproducible
Removes clutter from notebook so the notebooks can be focused on the scientific story rather than the code that accomplishes it

There is not much to modularise,
the only thing that this notebook does is implements a this allennlp module on the Cord dataset to generate the Sentence claims in sentence
had pushed it again, because the earlier output had a lot of clutter


fix go get for v2.0.0

when changing the major version of a go lib, go requires that the package path is changed as well (adding /v2 at the en in go.mod)
$ mkdir mnd-error                                                           [22:15:13]
$ cd mnd-error                                                              [22:15:21]
$ go mod init example.com/mnd-error                               [22:15:24]
go: creating new go.mod: module example.com/mnd-error

$ go get github.com/tommy-muehle/go-mnd                           [22:15:41]
go: downloading golang.org/x/tools v0.0.0-20190221204921-83362c3779f5
go: extracting golang.org/x/tools v0.0.0-20190221204921-83362c3779f5
go: finding golang.org/x/tools v0.0.0-20190221204921-83362c3779f5

$ cat go.mod                                                      [22:15:54]
module example.com/mnd-error

go 1.13

require github.com/tommy-muehle/go-mnd v1.3.0 // indirect <--- 1.3.0

$ go get<EMAIL_ADDRESS>                  [22:16:02]
go: finding github.com/tommy-muehle/go-mnd v2.0.0
go: finding github.com/tommy-muehle/go-mnd v2.0.0
go get<EMAIL_ADDRESS><EMAIL_ADDRESS>invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

@markus-wa I'm very sorry that I missed this PR in the past. But it's already fixed.

@tommy-muehle - no worries. Unfortunately I'm still getting the same error when trying go get<EMAIL_ADDRESS>or go get github.com/tommy-muehle/go-mnd/v2
go.mod still seems to be lacking /v2 on line 1 (https://github.com/tommy-muehle/go-mnd/blob/master/go.mod#L1)
any command that doesn't error only ever resolves to v1.3.0


Preproc redesign

Added table to navigate preproc images on each image
Dropdown link from amp in navbar to this table
Addresses issue #64

Works for me


Support for templates and expresions

When using dynamic paths inside require like:
require('shared/' + name)
//or
require(`shared/${name}`)

aliasing dosn't work. This is my propose to fix it, haven't any experience in writing babel plugins so isnt's sure is it ok.

Hi @wszerad, thanks for the contribution. I'll look more into it tonight, but please add tests and make sure the linting is correct ;)

Closing because of inactivity


Run phpcs check on build

Also fixes a new issue that was identified.
Refs #27

Also worth noting that tests/ is not checked on travis and contains a number of phpcs errors.


BUG Timed Loiter running indefinitely

PX4 1.8.0 - QGC 3.3.2 Android
A timed (30 sec.) loiter was planned just before the landing pattern. The loiter was entered and continued for 4 minutes before I had to manually select the next waypoint to go to.
I created an issue in QGC and was directed to open an issue in firmware.
https://logs.px4.io/plot_app?log=3ceb70c2-c26b-4c48-818c-0c1c7d31044

Possibly fixed in https://github.com/PX4/Firmware/pull/8883, but that PR is badly out of date.

This is another one to add to CI testing once it's been fixed.


Unable to install atom-terjs

Please help me to install atom-ternjs

it dose not  download Atom all the time shows there error i assume i need to doawnload an old virion can you help me to sort the problem out many thanks


running errors

I run exmples.ipynb, but received error message like this
No module named 'utils'
I found file utils.py at /data/ folder. Can you tell me why?
I am using python 3.5
Thanks

This work is built using Python 2.7. Maybe you can try it with Python 2.7.

@odek53r
I used $2to3 -w ***.py command to transfer your Python2 files into Python3 files, Then it works now. Thank you.


[State Restoration] Restorable FormField and TextFormField

Makes FormField, and TextFormField state restorable. An interesting situation with this is that FormField contains an internal generically typed _value. I'm not sure how to handle in a straightforward way since there's no guarantee that T is serializable. So to deal with this in TextFormField, I explicitly call setValue in restoreState to make sure that the internal value in FormFieldState is kept in sync with the _controller value whenever restoreState is called.
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Should be ready for another round of review!

The Google testing failures appear to be flakes. I reran them (internal link) and they're passing.


Literal Replace command works incorrectly when the replacement string contains '$'

Description of the problem or steps to reproduce
Basically I want to replace some variable name in my shell script with another variable name. In shell, to get a value of a variable, the syntax is '$variable'. So for example, i want to replace '$variableA' with '$variableB'.
I'm typing replace -l '$variableA' '$variableB' in micro's command line. micro correctly finds instances of '$variableA' but when i press Y to accept the replacement, what micro actually replaces with is empty string. So it basically deletes the found instance of '$variableA' instead of replacing it with '$variableB'.
I tried escaping the '$' sign in the replacement string with a backslash but then micro just replaces '$variableA' with the backslash.
This is probably not how it's supposed to work, especially when the 'replace' command is invoked in literal mode.
Specifications

Commit hash: c2cebaa3
OS: Debian Stretch
Terminal: xterm

Hm, in the moment someone knows what will be invoked in that moment then it becomes explainable, see https://pkg.go.dev/regexp#Regexp.Expand:

To insert a literal $ in the output, use $$ in the template.

Considering this in a possible fix would be easy in the first place, but may become quite complex in the moment every possible user combination needs to be considered in both modes. But as long as the non intuitive, but documented...

Note that search must be a valid regex (unless -l is passed). If one of the arguments does not have any spaces in it, you may omit the quotes.

...will be respected, then it remains simple:
diff --git a/internal/action/command.go b/internal/action/command.go
index 16d2fff8..0b88c3d7 100644
--- a/internal/action/command.go
+++ b/internal/action/command.go
@@ -785,6 +785,7 @@ func (h *BufPane) ReplaceCmd(args []string) {
        if noRegex {
                search = regexp.QuoteMeta(search)
        }
+       replaceStr = strings.ReplaceAll(replaceStr, "$", "$$")

        replace := []byte(replaceStr)

replace $$var $var1 -l does not work
nothing matches \$\$var

@237dmitry
For me it works. Do you have $$var present in your text? Because if not, then it's explainable. ;)

To insert a literal $ in the output, use $$ in the template

Do you have $$var present in your text?

Not I have not. I am trying to replace literal $var with $var1. Behavior is the same as in the issue:
replace $var $var1 -l    # replaced "$var" with space \u0020
replace \$var $var1 -l   # the same
replace $$var $var1 -l   # nothing matched \$\$var

regex replace is not working too. replace '\$var' '$var1'

For me it works.

Please show example

No I have not. I am trying to replace literal $var with $var1. Behavior is the same as in the issue:

Then the question is, why you tried to find $var with the search string $$var? ;)
As I tried to explain above:

Please follow the micro documentation, that the "search" string needs to be of a regex type or in case the literal search is used it can be of the normal literal type, but will be replaced implicit to regex type by micro...because the search is performed with regex. -l only controls the type how "search" needs to be interpret.
The the confusing or unexpected thing right now is, that the "replace" string is every time treat as literal.

regex replace is not working too. replace '$var' '$var1'

Isn't it?

Please show example

Here we go...
literal search:

regex search:

Did you realize that I linked a fix yesterday to use $ in replacements?
I agree, that it's ugly to add \\ to interpret special regex keywords as single character, but we do the same in the syntax YAMLs.
Maybe we could replace \ within the search string into \\ upon usage of explicit regex search, as we no do with the $ to $$ in the replace string (see #2954).

This does not work.
Literal search/replace:

Regex search/replace:

Same thing with quotes, double or single
micro is tar-compressed distributive from releases page
$ micro --version
Version: 2.0.12
Commit hash: c2cebaa3
Compiled on September 06, 2023

$ uname -rsov
Linux 6.5.5-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 23 Sep 2023 22:55:13 +0000 GNU/Linux

micro is tar-compressed distributive from releases page

And that's exactly the reason why I works for me, but doesn't for you. ;)
Have you read my following statement?

Did you realize that I linked a fix yesterday to use $ in replacements?

The PR #2954 was created 2 days ago and thus it can't be present in the last release. If you want this change, then you've to use a dedicated commit hash and build micro yourself or unfortunately you've to wait, till the PR will be merged into master and a new release done.

unfortunately you've to wait

I will wait. Not so often I use micro, mcedit is my favorite editor.

@237dmitry This is an open source project with a limited time that few people invest into developing it. Generally it's a good idea to avoid wasting their limited time on useless discussions.

Generally it's a good idea to avoid wasting their limited time on useless discussions.

You are right. I simply confirmed that the problem exists and needs to be fixed.

@JoeKar I'm not sure if this is the correct place to suggest this (should I open an new issue?) but after having played a bit more with the literal replace function, I feel that micro is very different in the way it implements this feature from every other text editor I've seen (well I've never touched vim or emacs but I'm well acquainted with nano and all sorts of editors for Windows and for Mac.
What I want (and I believe most users want that, as well) is just make one click, enter the text I want to replace, press Enter, enter the text I want to replace with, press Enter, get it replaced show the first instance, press enter and get it replaced and jump to the next instance. No frills and no need to think about how micro interprets whatever I type in. IMO the basic thing is to not have to enter the string-to-replace and the replacement-string into the same text field where you have to think about delimiters and special characters (which is also a guesswork) but rather just type in the string as is, just like this is implemented in Nano. Nothing should be interpreted - then this would actually be a literal replace, and a handy feature for arguably the vast majority of users.

@blunderful-scripts
I don't think, that an additional issue is necessary, because we've #2937 (and maybe even more opened). But to be honest, I fully confirm that the actual search/replace handling is a bit complicated an non-intuitive. Furthermore the actual search method (ignorecase) has further impact on the behavior.

well I've never touched vim or emacs

At least Vim does more or less the same within one command :s/foo/bar/ and in fact, it's a regular expression search.
The user definitely needs to understand, when a search is done in regex, that some special characters must be escaped to reflect the character instead of the special option/treatment behind it. It's the same with Vim, because it's the regex way to search.
So after #2954 what is now really missing? Do you want to have the literal search to be the default as in #2937? Because from your initial description you already wrote, that you invoked -l to do a literal search (and yes, you really discovered a bug in the replacement-string - handling), which should be the easiest approach. With this you don't need to care about further delimiters, you can even write without the quotes, just use a space between the two words and it isn't that many guesswork, since it's documented (commands.md).
If you would like to have a fully new search/replace menu/interface, then it probably would be worth a new feature request.
Side note:
Have a look at the open PRs and maybe how many from me are already open. Currently the zyedidia doesn't seem to be able to spent a lot of time into his project and the maintenance therefore is slowed down already and even if he could it would take months to sort out the actual issue and PR flood.

So after #2954 what is now really missing?

Personally, when editing code, I frequently need to match and replace a string that consists of arbitrary symbols, including spaces, quotation marks (double and single) and sometimes escaped special symbols. Trying to do that in regex mode means spending tons of time on carefully crafting an expression. Now I'm not sure how literal replace is actually supposed to work in micro, but as long as the string-to-replace and the replacement-string have to be entered sequentially into the same line, obviously the input has to be interpreted, which means there must be special symbols like spaces and quotation marks. If the string I want to replace contains some of these special symbols then I have to escape them in some way, with a syntax that is probably undocumented. Now what if the escaped symbol also happens to be a part of the string I want to replace? So you see, it's really easy to get to triple- or quadruple backslashes here, all that if the quotation marks and the spaces are the only special symbols that get interpreted. Which is possibly not the case, as this bug illustrates. So yeah, my dream is exactly the same behavior as in nano. Meaning separate input fields for string-to-replace and the replacement-string, and absolutely no interpretation that gets in the way.
I suppose regex replace could be useful for some very specific tasks but not as a go-to replace tool, at least not for me.

Do you want to have the literal search to be the default as in #2937?

I'm fine with either default, as long as I can set my own default in the config file.  Not a big deal for me but the question is, how big of a deal it is for the average user in your target audience? My opinion is that best practice when designing a tool is to make it so the user won't need to rtfm, at least for the core functionality, and replace is definitely a core function in a text editor.

Ok, let us see, what can be done in the near future:

additional option to control the default search mode (literal or regex)
maybe additional option to control the default search interface (replace 'search' 'value' or multi-line input prompt)

In case my summarize is correct, please create a new issue with this + possible the stuff I possibly missed.


Syntastic has stopped working with<EMAIL_ADDRESS>

Which makes me sad because now it seems like most of the ES6 stuff I use is handled properly. ;_;
I updated Syntactic to HEAD, and have verified that standard is working by running :!standard against an erroneous file from inside vim. If nothing obvious occurs to you about how the formatting might have changed in beta.4, I can dig into this more tomorrow.

The fix for me was to use io.js@>=1.

@yoshuawuyts
bauchelain% iojs --version
v1.5.1

This is probably an eslint issue, because the diff between beta3 and beta4 is pretty much entirely the change to<EMAIL_ADDRESS>

I can't think of anything that might have caused this, except maybe a change in Syntastic. Does 3.0.0-beta3 work fine with Syntasic HEAD?

Does 3.0.0-beta3 work fine with Syntastic HEAD?

It sure does! I didn't try changing the Syntastic version until after beta4 stopped working, but even then, I can switch between beta3 and beta4 and it starts and stops working accordingly. I really need this to work, so I'm going to dig in a bit further.

Hey @othiym23, can you try again with v3.2.0? I removed automatic detection of stdin, which is error-prone and may have been confusing Syntastic/vim.

So, to capture here what we discussed on IRC, when I go into bin/cmd.js and comment out the line that sets the default value for stdin to !process.stdin.isTTY, everything starts magically working again. It's probably best to require users to explicitly call --stdin, or follow browserify's lead and observe the Unix convention of using - as the way of indicating that input should come from stdin.

Yep, that change is already released as 3.2.0!
On Wed, Mar 18, 2015 at 10:11 PM Forrest L Norvell<EMAIL_ADDRESS>wrote:

So, to capture here what we discussed on IRC, when I go into bin/cmd.js
and comment out the line that sets the default value for stdin to
!process.stdin.isTTY, everything starts magically working again. It's
probably best to require users to explicitly call --stdin, or follow
browserify's lead and observe the Unix convention of using - as the way
of indicating that input should come from stdin.
—
Reply to this email directly or view it on GitHub
https://github.com/feross/standard/issues/65#issuecomment-83325082.

Oh, I see you were just clarifying for posterity.

Yep<EMAIL_ADDRESS>works again! :tada:


Desktop client ignores "--injector-credentials" option

When testing commit 276c22d3 on GNU/Linux Docker with credentials in the injector, adding the exact same value for the injector-credentials in ouinet-client.conf doesn't seem to have effect in the client, and the browser keeps on asking authentication for the proxy until it is entered. Completely disabling the option in the client yields the same result. After the correct credentials are entered at the browser, everything seems to work as expected.

Fixed in commit 6aa7ef455


archive/zip: reject certain invalid archives

This is not a defect report, but a suggestion to review the Zip archive reader with this new and authoritative zipbomb treatise in mind:
https://www.bamsoftware.com/hacks/zipbomb/

CC @dsnet

Hi @MichaelTJones, thanks for the report. This attack depends on two properties (called "insights" in the paper).
I believe Go's zip.Writer refuses to encode zip archives like this. However, Go's zip.Reader is probably susceptible to both properties mentioned by insight 1 and 2 (I did not verify). Insight 2 seems to be obviously an invalid ZIP archive, and our reader should error on such files. However, it's not clear to me that the directly overlapping files of insight 1 is invalid. I personally believe this should be an error as well, but I'm concerned that there may be people actually depending on this property.

I read the whole paper. The other point was that in addition to specific
defenses, a careful implementation needs limits on time and space. That
seemed good to me--if the expansion passes 1 GiB and the user's not set a
more liberal upper limit, then bail.
I do not believe (but do not know for sure) that there are ever valid cases
for overlapping file bodies. I'm thinking that this should either be
prohibited or like file size limit, be an option that defaults to disabled.
On Wed, Jul 10, 2019 at 10:49 AM Joe Tsai<EMAIL_ADDRESS>wrote:

Hi @MichaelTJones https://github.com/MichaelTJones, thanks for the
report. This attack depends on two properties (called "insights" in the
paper).
I believe Go's zip.Writer refuses to encode zip archives like this.
However, Go's zip.Reader is probably susceptible to both properties
mentioned by insight 1 and 2 (I did not verify). Insight 2 seems to be
obviously an invalid ZIP archive, and our reader should error on such
files. However, it's not clear to me that the directly overlapping files of
insight 1 is invalid. I personally believe this should be an error as well,
but I'm concerned that there may be people actually depending on this
property.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/33026?email_source=notifications&email_token=AB4DFJKRBT2ATW2QVG3JJ6DP6YOMJA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZUHECI#issuecomment-510161417,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB4DFJLFRXAUN7X52OB22H3P6YOMJANCNFSM4H7QSYGQ
.

--
Michael T<EMAIL_ADDRESS><EMAIL_ADDRESS>

I read that article when it came out, and while it's extremely interesting in its techniques, I agree with the point it makes that unpacking archives with untrusted sources requires CPU, time and memory limits.
We should fix archive/zip if it accepts invalid archives, but I don't think we can make it safe to run on arbitrary archives without resource limits. Offering an API to easily apply limits on the unpacking process would be interesting, but a separate issue and definitely not for Go 1.13.
Aside from the insights, the table also points out other invalid zips that we decode, so we should probably also fix those.
/cc @rsc since he's credited on that article, and might have relevant insight.

I'll only add that lately I've been processing an archive (RSC's Go Corpus,
split, bundled, compressed, an archived that is 752 MB when decompressed,
so the defaults might be pretty big -- a few seconds, a few GB, etc. --
rather than cater to MIME-type decompression sizes.
On Wed, Jul 10, 2019 at 12:07 PM Filippo Valsorda<EMAIL_ADDRESS>wrote:

I read that article when it came out, and while it's extremely interesting
in its techniques, I agree with the point it makes that unpacking archives
with untrusted sources requires CPU, time and memory limits.
We should fix archive/zip if it accepts invalid archives, but I don't
think we can make it safe to run on arbitrary archives without resource
limits. Offering an API to easily apply limits on the unpacking process
would be interesting, but a separate issue and definitely not for Go 1.13.
Aside from the insights, the table also points out other invalid zips that
we decode, so we should probably also fix those.
/cc @rsc https://github.com/rsc since he's credited on that article,
and might have relevant insight.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/33026?email_source=notifications&email_token=AB4DFJMWNDMTS4TYA3KESPDP6YXOXA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZUNYWA#issuecomment-510188632,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB4DFJND6DLJKKPH74LGREDP6YXOXANCNFSM4H7QSYGQ
.

--
Michael T<EMAIL_ADDRESS><EMAIL_ADDRESS>

Is anyone working on this? If not I'd like to take it.

There is more here: https://github.com/golang/go/issues/33036
It is not necessary to address it everywhere at once by the same mechanism,
but it does seem important to approach zip decoding as an instance of the
general issue. (As in maybe there is a "limit" type passed as an optional
argument, limit having setters & setters but opaque contents, and thus the
ability to change implementation details later as more kinds of encoding
are protected this way.) That's just an idea for consideration.
Nobody "has" this at present.
On Mon, Jul 15, 2019 at 5:29 PM Norman B. Lancaster <
<EMAIL_ADDRESS>wrote:

Is anyone working on this? If not I'd like to take it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/33026?email_source=notifications&email_token=AB4DFJMEA3JMRLPJZV2FE4LP7UI6FA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ7KTCY#issuecomment-511617419,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB4DFJOPPNRGHSRJNSA73LDP7UI6FANCNFSM4H7QSYGQ
.

--
Michael T<EMAIL_ADDRESS><EMAIL_ADDRESS>

A couple of things to watch out for here -- reports of in-the-wild zip files that could no longer be processed after Debian merged a patch to reject overlapping files (#931433: unzip: CVE-2019-13232).

#931895: unzip: zip bomb false positives in Java ecosystem
The problem here seems to be duplicate central directory headers that point to the same local file header, which shows up as overlapping files. See particularly Message #73. See also https://github.com/gradle/gradle/issues/9990.
#932404: firefox-esr, FTBFS "possible zip bomb".
The issue here is Firefox's omni.ja file, which uses a nonstandard zip layout with the central directory near the beginning, not the end. This has caused plenty of problems over the years (e.g. Mozilla #595473, #605524, #775568), but archive/zip can nevertheless handle it currently.

Do we want to support FireFox's non-standard zip layout? The current implementation of zip.Reader searches for a CDE in the last 64KB of the file.
Regarding duplicate CDE's, they can be scrubbed prior to scanning for overlapping files.

I don't know of a prevailing philosophy beyond robust, general, and least
surprise.
It seems that "avoiding disaster" and "spec compliance policing" are
different goals. GCC/C++ has the notion of "strict" for such cases. Ian and
Rob are likely the best to opine here.
On Mon, Jul 22, 2019 at 6:56 AM Norman B. Lancaster <
<EMAIL_ADDRESS>wrote:

Do we want to support FireFox's non-standard zip layout? The current
implementation of zip.Reader searches for a CDE in the last 64KB of the
file.
Regarding duplicate CDE's, they can be scrubbed prior to scanning for
overlapping files.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/33026?email_source=notifications&email_token=AB4DFJIAG7YP42C4VKUSUBLQAW4AFA5CNFSM4H7QSYG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2P7TGA#issuecomment-513800600,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB4DFJPT2YHJBHRVSJX62G3QAW4AFANCNFSM4H7QSYGQ
.

--
Michael T<EMAIL_ADDRESS><EMAIL_ADDRESS>

I am just about ready to submit the CL for the zip bomb detection. I am going to go ahead and push the CL with full tests including the large binary files for review. If they need to be removed I can take care of it on the CL.

Do we want to support FireFox's non-standard zip layout? The current implementation of zip.Reader searches for a CDE in the last 64KB of the file. This means that although small test files encoded as described in the linked blog post might work, once the blob of data following the CDE grows to 64KB-(size of CDE) it will stop working.

No, zip.Reader searches for the EOCD in the last 64 KB. The EOCD contains a pointer to the central directory, which may be anywhere in the file. zip.Reader does not currently have any problem with the Firefox layout, even if the zip file is larger than 64 KB.

Do we want to support FireFox's non-standard zip layout? The current implementation of zip.Reader searches for a CDE in the last 64KB of the file. This means that although small test files encoded as described in the linked blog post might work, once the blob of data following the CDE grows to 64KB-(size of CDE) it will stop working.

No, zip.Reader searches for the EOCD in the last 64 KB. The EOCD contains a pointer to the central directory, which may be anywhere in the file. zip.Reader does not currently have any problem with the Firefox layout, even if the zip file is larger than 64 KB.

Thank you for the clarification!

Change https://golang.org/cl/187357

// Check for overlapping file bodies (Better Zip Bomb)
for il := range z.File {
	l := z.File[il]
	lstart := uint64(l.headerOffset)
	lend := uint64(l.headerOffset) + l.CompressedSize64
	for ir := il + 1; ir < len(z.File); ir++ {
		r := z.File[ir]
		rstart := uint64(r.headerOffset)
		rend := uint64(r.headerOffset) + r.CompressedSize64
		if lend <= rstart || lstart >= rend {
			continue
		}
		return ErrInvalid
	}
}

This looks like an O(n2) algorithm. You should test whether its performance is adequate on a zip file containing many files, such as the ffff.zip from https://github.com/thejoshwolfe/yauzl/issues/108. You may be better off first sorting the array of Files (or indices into the array) by their headerOffset. After that, you only need a linear scan to check whether File n overlaps File n+1.
lend := uint64(l.headerOffset) + l.CompressedSize64 is the wrong calculation because it ignores the size of the local file header itself, which is variable because of the filename and extra field. I think what you want is lend := l.DataOffset() + l.CompressedSize64 (except that you need to check for error from DataOffset).
The additions probably have integer overflow issues. You should test with some Zip64 files that are hacked to have sizes close to 264.
There's no need to include the entire zbsm.zip–and doing so is probably a bad idea, as its hash has no doubt been added to malware detection engines by now. You can generate your own smaller test case using the source code.
$ ./zipbomb --mode=quoted_overlap --num-files=3 --compressed-size=234 > test.zip
$ wc -c test.zip
500 test.zip
$ ./ratio test.zip
test.zip	678120 / 500	1356.24	+31.323 dB

I will say, though, that in my opinion the motivation for this patch is misguided. Its most likely outcome is breaking compatibility with some genuine zip files, without doing much to protect callers against zip bombs. Even a naive non-overlapped DEFLATE bomb can have a compression ratio over 1000, which will be too much for some applications.

Just in case it will help with crafting this fix, https://github.com/ronomon/zip has mitigations for overlapping local file headers, as well as more "obviously bad data".


Heuristic evalution

Heuristic evaluation of freeCodeCamp’s general website and more specifically the homepage and signup.
Checklist:

[] I have read freeCodeCamp's contribution guidelines.
[] My pull request has a descriptive title (not a vague title like Update index.md)
[] My pull request targets the main branch of freeCodeCamp.
[] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

Hey @Telavivken
Thank you for opening this pull request.
This is a standard message notifying you that we've reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you.
Thank you and happy coding.


最新版本tidb与xorm编译出错...

# github.com/go-xorm/tidb
../github.com/go-xorm/tidb/tidb_dialect.go:140: db.Logger undefined (type *tidb has no field or method Logger)
../github.com/go-xorm/tidb/tidb_dialect.go:141: db.Logger undefined (type *tidb has no field or method Logger)
../github.com/go-xorm/tidb/tidb_dialect.go:252: db.Logger undefined (type *tidb has no field or method Logger)
../github.com/go-xorm/tidb/tidb_dialect.go:253: db.Logger undefined (type *tidb has no field or method Logger)
../github.com/go-xorm/tidb/tidb_dialect.go:282: db.Logger undefined (type *tidb has no field or method Logger)
../github.com/go-xorm/tidb/tidb_dialect.go:283: db.Logger undefined (type *tidb has no field or method Logger)

@insionng  Thanks for your report! I will fix this.


AMS Case Study Update

Working on some cleanup and updating given the evolving MetPy API. @kgoebber and @dopplershift please push your changes as after Thursday I won't be doing much here until after New Years.

[x] Flake notebook and solutions
[x] Update to .format style
[ ] Make notebook run cleanly in CI
[x] Make user fill in templates match solution updates
[ ] Replace v_vorticity et al with new names from MetPy 0.8 (to be released).
[x] Add standard logo/header

Still failing because of gradient, but that should get fixed up with MetPy 0.7. Feel free to merge or push directly.


cannot scroll to older messages

i'm unable to load older messages in a conversation. there is a limited scrolling up.

please fix it as soon as possible

Fixed by commit "44863b711998b4b01e4ce233f5bdc64dfaa042a8"
Launch IgDM and let the AutoUpdater do the job, you'll get the new feature ;)

this issue is fixed on the version 2.5.1 please visit the site http://igdm.me to download the latest version


Introduction SDEs/SDOs description

encryption of secret keys is advised but there is nothing said to decryption process where and how this process can occur and if there is s.th. or someone that who could decrypt those apparently important keys in case of eventual problems or missing conditions for decryption. This is elementary information particularly for root key use case.

It's not clear which section we are talking here.
In section 2.2.4 we list security attributes for SDOs.  Although I like this concept, I was not comfortable with enumerating a list of security attributes for a couple of reasons.  1) we cannot list all possible security attributes, 2) we shouldn't mandate a list of security attributes since different contexts will have different security requirements.  But, I'm happy to leave this list as an example, and even happier if we annotate as an example.
I found NIST SP 800-152, Section 6.2.1 "Metadata Elements" under Section 6.2 "Key Metadata".  The Security Attributes can be mapped the Key Metadata, even though not all SDO are keys:
SDO.ID -> 6.2.1.b Key Identifier
SDO.Type -> 6.2.1.l Key Type
SDO.AuthData -> 6.2.1.s.iv [non-crypt trust process for key protection]
SDO.ReAuth -> 6.2.1.t.iv [non-crypt trust process for metadata protection]
SDO.Conf -> 6.1 [cryptographic key]
SDO.Export -> 6.2.1.n [key security policy] and 6.2.1.s.ii [non-crypt trust process for key protection]
SDO.Integrity -> 6.2.1.s.i [key integrity]
SDO.Bind ->  6.2.1.n [key security policy] and 6.2.1.s.ii [non-crypt trust process for key protection]
The NIST document has a lot of nice options in section 6.2.1 for SDO security attribute "metadata".  I'm open to referencing this document without copying it contents into the cPP.
As for SDOs for which confidentiality is called for, then anytime a private key (in the case of asymmetric cryptographic algorithms), or secret key (in the case of symmetric cryptographic algorithms) is used to encrypt or decrypt the protected element (which we are calling the SDE to distinguish it from the other security attributes/metadata of the SDO), then those operations should only be carried out within the confines of an approved DSC.
Perhaps we add the following wording to the assumption A.TRUSTED_PEER:  Hosts to the DSC and client applications are assumed to be trustworthy and will not transfer private keys (in the case of asymmetric cryptographic algorithms) or secret key (in the case of symmetric cryptographic algorithms) to unauthorized or untrustworthy entities.
There are certain root keys (private or secret) which cannot be encrypted since they may exist in a boot environment in which it's not possible to supply the necessary key to decrypt it.  I'm not sure that we need to say a lot about this unless we are prepared to levy hardware requirements for protected cleartext keys.
The second paragraph of section 2.4.6 talks about SDOs with wrapped keys; encryption is implied here.  This cPP sets up mechanisms to encrypt objects such as keys before exporting from the DSC, as well as setting up encrypted channels in those cases in which keys are to be transported without wrapping.  Ultimately, the security policy set up with the key when it was provisioned determines whether or not to wrap it before exporting or transporting; the DSC should honor those security policies.

OK, so the initial question here, as I see it, is more about what is meant by the various SDEs and SDOs. I think the confusion comes into play when it is stated that root keys are administrator keys. This is true in a sense, but I don't think it is the right way to put this. There are RoT keys and other root keys (best example would probably be a hardware-fused encryption key, but could also encompass say a fused hash for a digital signature to verify firmware) which may be used by the DSC, but which aren't tied to authentication as they are the underlying keys of the system (on which all the security may be built). Some of these would be used internally regardless of the role as they are how the system works.
So from that standpoint I think that stating things about root keys should be taken out the description. Even the diagram itself shows root keys being separate from the authorization, so I think this just confuses things.
@slpotte the rest of the point about changing the SFR is valid, I don't know if it is a priority right now, but it could be done as a separate topic.

Which section?

Section 2.4.4,
https://github.com/DSC-iTC/cPP/blob/e1cb23c5e74585d10b155a6d427b9e10376a708a/cPP_DSC.adoc?plain=1#L338

So the action is:
Remove "Root keys installed by the manufacturers are considered administrator key material." from the middle of the second paragraph.
I saw nothing about SDOs and SDEs in section 2.4.4.


Extend nx type

Extend: NX type by NX_CHAR_OR_INT
Include XRD in the list of plugin tests.

LGTM.
Will we have to change the np types when going to numpy v2?

Currently no need. As nomad is pinned around 1.26. Let what comes in future.


Authentication failed while logging on https://capella.ertms.be/

Hie,
I try to log on the above server.
I have access to:
Image has been removed.
I get to this point:
Image has been removed.
I copy my 'T4CLogin password'.
Before I get to use it, I get following error message:
Image has been removed.
I get this error for both Capella 5.2.0 and 6.0.0.
Prior to this, I used to be asked for my authentication.
What am doing wrong ?

Hi @PLeMag, thanks for the request. Please don't expose internal information in our Github project board. Please write an email to<EMAIL_ADDRESS>so that we can investigate your authentication issue. Thanks!