Fix processor and parser PUBSUB tests to check for logger err msgs

Initially, we returned errors from the processor/parser through the PUBSUB flow. However, this causes the guacone ingestor to stall (https://github.com/guacsec/guac/issues/763) unintentionally. The fix is to not return an error when processing a doc happens (https://github.com/guacsec/guac/pull/764)...
However, this makes the tests fail as it no longer returns for an error to check for. This should be changed so that a test logger can be passed in context to see if an error has occurred.
See https://github.com/guacsec/guac/pull/764/files for commented out tests to fix

Do we want to implement something like this for our tests https://github.com/sirupsen/logrus/blob/master/hooks/test/test_test.go#L16? Just curious.

ooo that would be really neat!

@lumjjb do we still want to keep the [processor: uuid] part of the logs? Might be a hinderance to assert the logged string since uuid gets generated within the method.

Upon discussion with @lumjjb seems like in the long term, it would be preferred to have a separate function for the test logger that would be forced to run by setting some flags. Scrapping the idea of using logrus or something similar for now.

I can take this one.

@lumjjb @stevemenezes As a fix for this issue, I am trying to insert a zap logger into the ctx, and retrieve it in Subscribe function. When the subscribe function returns in the testing function, I check if the log messages contain any common keyword between all the error messages. Please let me know if this sounds like a good fix. Thanks!


Minor tweaks to toast message when deleting multiple plants.

Coverage remained the same at 90.8% when pulling 5d6d2815f11bc88e299ba47271c67602ed1b1976 on RickCarlino:batch_delete_toasts_idea_ii into b98944e70d01dc4f691ae44c8fac4062913ddf26 on FarmBot:staging.


Warning when compiled by GCC 5.1 upwards with -Wabi-tag

#include <snowhouse/snowhouse.h>
int main()
{
  return 0;
}

When code above is compiled by GCC 5.1 upwards with -Wabi-tag, compilation throws many errors like this:
In file included from ../snowhouse/fluent/operators/constraintoperator.h:10:0,
                 from ../snowhouse/fluent/fluent.h:12,
                 from ../snowhouse/snowhouse.h:26,
                 from a.cc:1:
../snowhouse/fluent/operators/invalidexpressionexception.h:12:10: warning: ‘snowhouse::InvalidExpressionException’ does not have the "cxx11" ABI tag that ‘std::__cxx11::string {aka std::__cxx11::basic_string<char>}’ (used in the type of ‘snowhouse::InvalidExpressionException::m_message’) has [-Wabi-tag]
   struct InvalidExpressionException
          ^~~~~~~~~~~~~~~~~~~~~~~~~~
../snowhouse/fluent/operators/invalidexpressionexception.h:23:17: note: ‘snowhouse::InvalidExpressionException::m_message’ declared here
     std::string m_message;

It was caused by Dual ABI (introduced in GCC 5.1) which changes std::string implementation among other changes. Some other pinpoints follows:

see last paragraph in Troubleshooting on https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html - it describes issues on snowhouse - methods returing std::string or class having member variable of type std::string.
case (c) on https://greek0.net/blog/2016/10/29/gcc_compatibility/ - this article describes how it works
on some mailing thread I found answer which proves my opinion about -Wabi-tag option, it look likewise:
Q: "Why -Wall or -Wextra doesn't include -Wabi-tag by default?" >> A: "Because it is useful mainly for library vendors"

The abi-tag option checks only member variables and returned types of functions/methods which are not tagged, other usage of std::string is OK
For example the std::string in param corresponds to std::__cxx11::string (from inline namespace defined in libstdc++) and this type has different mangle name than std::string. But the warning informs that cxx11 is inherited for returned type as well.
Affected classes with member variables of type std::string and returning std::string from methods could be "annotated" by _GLIBCXX_DEFAULT_ABI_TAG to get rid of these warnings. See /usr/lib/...../bits/c++config.h where _GLIBCXX_DEFAULT_ABI_TAG macro is defined as alias to _GLIBCXX_ABI_TAG_CXX11 which refers to '_attribute ((abi_tag ("cxx11")))'.
Warnings shouldn't be thrown here IMHO, but the question is how to fix it?

Thank you very much. I'm hearing about this the first time and I wonder if this is actually a problem since snowhouse/bandit are header-only libraries and hence always compiled with the same ABI as the user code. In other words: the warning does IMHO not make sense for the code at all, so I am also unwilling to tag the code.
However, I see that if you are using snowhouse/bandit for your library tests and you use -Wabi-tag for your whole code base (including tests), this is a problem. One solution would be to declare bandit/snowhouse as system headers with the -isystem option. Maybe I get better ideas.
Also note that bandit (but not snowhouse) is C++11-only, so setting -D_GLIBCXX_USE_CXX11_ABI=1 in the CXXFLAGS is safe. If I understood the scenario correctly, this should silence the warnings. (However, on my g++7, setting it to 0 silences the warnings, setting it to 1 does not... So: for a C++11-only setting, do not use -Wabi-tag?)

Thank you for reply.
Generally you are right, -Wabi-tag is useless for header-only libs, for building final app binary as well.
There is probably only one case where -Wabi-tag give benefits to us - building a library.
In our case we want -Wabi-tag option enabled because we use this code on wrapping library which is not header-only and it can cause potential issues on linking.
Thanks for -isystem option, I didn't know that. It can help me to figure out how to avoid useless warnings from snowhouse.
I think you can close this issue as Won't fix.


Builders: PostgreSQL builder

doesn't follow builders guidlines
not complete

Required by #492.

I'm facing an error where builder profile got removed
j.builder.db.postgres.build(reset=True)

then i got
+ cd /tmp/builders/psql
+ . /tmp/builders/psql/env.sh
/tmp/builders/psql/builder_postgresql.sh: line 3: /tmp/builders/psql/env.sh: No such file or directory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/sandbox/lib/jumpscale/Jumpscale/builder/system/BuilderBaseClass.py", line 108, in wrapper_action
    res = self.apply_method(func, kwargs)
  File "/sandbox/lib/jumpscale/Jumpscale/builder/system/BuilderBaseClass.py", line 63, in apply_method
    return func(**kwargs)
  File "/sandbox/lib/jumpscale/Jumpscale/builder/db/BuilderPostgresql.py", line 28, in build
    self._execute(cmd)
  File "/sandbox/lib/jumpscale/Jumpscale/builder/system/BuilderBaseClass.py", line 323, in _execute
    showout=showout,
  File "/sandbox/lib/jumpscale/Jumpscale/sal/process/SystemProcess.py", line 103, in execute
    replace=replace,
  File "/sandbox/lib/jumpscale/Jumpscale/core/InstallTools.py", line 1175, in execute
    raise RuntimeError(msg)
RuntimeError:
Could not execute:    bash /tmp/builders/psql/builder_postgresql.sh

stderr:
    + cd /tmp/builders/psql
    + . /tmp/builders/psql/env.sh
    /tmp/builders/psql/builder_postgresql.sh: line 3: /tmp/builders/psql/env.sh: No such file or directory

Could not execute:    bash /tmp/builders/psql/builder_postgresql.sh

stderr:
    + cd /tmp/builders/psql
    + . /tmp/builders/psql/env.sh
    /tmp/builders/psql/builder_postgresql.sh: line 3: /tmp/builders/psql/env.sh: No such file or directory

The cause of the error mentioned above is that j.builder.tool.file_download will remove the entire destination if expand=True

Fixed, the build is working now (using version 9.6.13).
Will check install() then we should have a working setup.

Branch: development_builder
build/install/start/stop should be working now.

j.builder.db.postgres.build() in fresh installation.
stderr:
    + cd /sandbox
    + . /sandbox/env.sh
    ++ export LC_ALL=en_US.UTF-8
    ++ LC_ALL=en_US.UTF-8
    ++ export LUALIB=/sandbox/openresty/lualib
    ++ LUALIB=/sandbox/openresty/lualib
    ++ export LANG=en_US.UTF-8
    ++ LANG=en_US.UTF-8
    ++ export 'LUA_PATH=?.lua;/sandbox/openresty/lualib/?/init.lua;/sandbox/openresty/lualib/?.lua;/sandbox/openresty/lualib/?/?.lua;/sandbox/openresty/lualib/?/core.lua;/sandbox/openr
        esty/lapis/?.lua'
    ++ LUA_PATH='?.lua;/sandbox/openresty/lualib/?/init.lua;/sandbox/openresty/lualib/?.lua;/sandbox/openresty/lualib/?/?.lua;/sandbox/openresty/lualib/?/core.lua;/sandbox/openresty/la
        pis/?.lua'
    ++ export PBASE=/sandbox
    ++ PBASE=/sandbox
    ++ export 'LUA_CPATH=/sandbox/openresty/lualib/?.so;./?.so'
    ++ LUA_CPATH='/sandbox/openresty/lualib/?.so;./?.so'
    ++ export PYTHONHTTPSVERIFY=0
    ++ PYTHONHTTPSVERIFY=0
    ++ export HOME=/root
    ++ HOME=/root
    ++ export HOST=faf7e1be4628
    ++ HOST=faf7e1be4628
    ++ export PYTHONPATH=/sandbox/lib:/sandbox/lib/jumpscale:/sandbox/lib:/sandbox/lib/jumpscale:
    ++ PYTHONPATH=/sandbox/lib:/sandbox/lib/jumpscale:/sandbox/lib:/sandbox/lib/jumpscale:
    ++ export PATH=/sandbox/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    ++ PATH=/sandbox/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    ++ export 'PS1=3BOTDEVEL:faf7e1be4628:\W: '
    ++ PS1='3BOTDEVEL:faf7e1be4628:\W: '
    ++ export LAPIS_OPENRESTY=/sandbox/bin/openresty
    ++ LAPIS_OPENRESTY=/sandbox/bin/openresty
    + cd /tmp/builders/psql
    + ./configure --prefix=/sandbox/apps/pgsql --bindir=/sandbox/bin --sysconfdir=/sandbox/cfg --libdir=/postgres --datarootdir=/sandbox/apps/pgsql/share
    configure: WARNING:
    *** Without Bison you will not be able to build PostgreSQL from Git nor
    *** change any of the parser definition files.  You can obtain Bison from
    *** a GNU mirror site.  (If you are using the official distribution of
    *** PostgreSQL then you do not need to worry about this, because the Bison
    *** output is pre-generated.)
    configure: WARNING:
    *** Without Flex you will not be able to build PostgreSQL from Git nor
    *** change any of the scanner definition files.  You can obtain Flex from
    *** a GNU mirror site.  (If you are using the official distribution of
    *** PostgreSQL then you do not need to worry about this because the Flex
    *** output is pre-generated.)
    + make
    copy_fetch.c:161:1: error: conflicting types for ‘copy_file_range’
     copy_file_range(const char *path, off_t begin, off_t end, bool trunc)
     ^~~~~~~~~~~~~~~
    In file included from copy_fetch.c:16:0:
    /usr/include/unistd.h:1110:9: note: previous declaration of ‘copy_file_range’ was here
     ssize_t copy_file_range (int __infd, __off64_t *__pinoff,
             ^~~~~~~~~~~~~~~
    make[3]: *** [copy_fetch.o] Error 1
    make[2]: *** [all-pg_rewind-recurse] Error 2
    make[1]: *** [all-bin-recurse] Error 2
    make: *** [all-src-recurse] Error 2

j.builder.db.postgres.install()
Could not execute: 'bash /sandbox/builder_postgresql.sh'stderr:
    + cd /sandbox
    + . /sandbox/env.sh
    ++ export LC_ALL=en_US.UTF-8
    ++ LC_ALL=en_US.UTF-8
    ++ export LUALIB=/sandbox/openresty/lualib
    ++ LUALIB=/sandbox/openresty/lualib
    ++ export LANG=en_US.UTF-8
    ++ LANG=en_US.UTF-8
    ++ export 'LUA_PATH=?.lua;/sandbox/openresty/lualib/?/init.lua;/sandbox/openresty/lualib/?.lua;/sandbox/openresty/lualib/?/?.lua;/sandbox/openresty/lualib/?/core.lua;/sandbox/openr
        esty/lapis/?.lua'
    ++ LUA_PATH='?.lua;/sandbox/openresty/lualib/?/init.lua;/sandbox/openresty/lualib/?.lua;/sandbox/openresty/lualib/?/?.lua;/sandbox/openresty/lualib/?/core.lua;/sandbox/openresty/la
        pis/?.lua'
    ++ export PBASE=/sandbox
    ++ PBASE=/sandbox
    ++ export 'LUA_CPATH=/sandbox/openresty/lualib/?.so;./?.so'
    ++ LUA_CPATH='/sandbox/openresty/lualib/?.so;./?.so'
    ++ export GOPATH=/sandbox/go_proj
    ++ GOPATH=/sandbox/go_proj
    ++ export PYTHONHTTPSVERIFY=0
    ++ PYTHONHTTPSVERIFY=0
    ++ export HOME=/root
    ++ HOME=/root
    ++ export GOROOT=/sandbox/go
    ++ GOROOT=/sandbox/go
    ++ export HOST=57e8e0ac844f
    ++ HOST=57e8e0ac844f
    ++ export PYTHONPATH=/sandbox/lib:/sandbox/lib/jumpscale:/sandbox/lib:/sandbox/lib/jumpscale:
    ++ PYTHONPATH=/sandbox/lib:/sandbox/lib/jumpscale:/sandbox/lib:/sandbox/lib/jumpscale:
    ++ export PATH=/sandbox/go/bin:/sandbox/go_proj/bin:/sandbox/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    ++ PATH=/sandbox/go/bin:/sandbox/go_proj/bin:/sandbox/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
    ++ export 'PS1=3BOTDEVEL:57e8e0ac844f:\W: '
    ++ PS1='3BOTDEVEL:57e8e0ac844f:\W: '
    ++ export LAPIS_OPENRESTY=/sandbox/bin/openresty
    ++ LAPIS_OPENRESTY=/sandbox/bin/openresty
    + cd /sandbox
    + mkdir -p log
    + mkdir -p /sandbox/apps/psql/data
    + chown -R postgres /sandbox/apps/psql/data
    + sudo -u postgres /sandbox/bin/initdb -D /sandbox/apps/psql/data -E utf8 --locale=en_US.UTF-8
    /sandbox/builder_postgresql.sh: line 8: sudo: command not found

After install sudo
JSX> j.sal.process.getProcessPid('postgres')
[]

JSX> j.builder.db.postgres.start()
Sun 26 11:03:23 StartupCMD.py    - 121 - startupcmd:startupcmd              : IS RUNNING postgres
JSX> j.builder.db.postgres.stop()
Sun 26 11:03:28 StartupCMD.py    -  63 - startupcmd:startupcmd              : stop:
postgres
Sun 26 11:03:28 StartupCMD.py    -  70 - startupcmd:startupcmd              : found process to stop:28602
Sun 26 11:03:29 StartupCMD.py    - 102 - startupcmd:startupcmd              : IS HALTED postgres
JSX> j.sal.process.getProcessPid('postgres')
[28603, 28605, 28606, 28607, 28608, 28609]

It didn't stop
Builder doesn't have sandbox or test.

@AhmedHanafy725 i cannot see what file this builder is downloading?
Are you sure this test was on a fresh installation? because that compilation error should only appear if you've an old source directory (before 9.6.13).
Also, this i updated this builder because we only need it for #492, sandbox() would take more time.

@abom for the fresh installation it didn't have your commit fix this and it works fine now.

It didn't stop

If startup cmds cannot handle this, should i implement my own stop commands?

If startup cmds cannot handle this, should i implement my own stop commands or it would be an issue in startup cmds?

yes, If you need special stop you can override stop method in your builder

Fixed start/stop by using pg_ctl, also added sandbox().

Tried to create an flist and start it, but seems like some libraries are missing, maybe something is still wrong with j.sandboxer.libs_sandbox() or something.
Flist: https://hub.grid.tf/abdelrahman/psql.flist.md
from
from zeroos.core0.client import Client

cl = Client('node_ip_goes_here')
cont_cl = cl.container.client(container_id_goes_here)
cont_cl.system('/sandbox/bin/pg_ctl start -D /sandbox/apps/psql/data',env={'LD_LIBRARY_PATH':'/sandbox/lib:/lib'}).get()

Output:

STATE: 0 ERROR
STDOUT:

STDERR:

DATA:
fork/exec /sandbox/bin/pg_ctl: no such file or directory

Will try to set some env variables like PATH to see if it's the reason (to resolve https://github.com/threefoldtech/jumpscaleX/issues/527#issuecomment-497368647).

I'm trying to validate j.tools.sandboxer, by creating an flist from a known working container (digitalmex builder).
Will compare the output of the flist with a working one to see libs_sandbox() still working.

While building digitalmex, capnp builder is broken, tests fail
test-suite.log

Sandbox now working after fixing j.sandboxer, now i'm trying to make the flist autostart works.

Finally, seems like the problem was zero-os parsing toml!
zero-os uses https://github.com/naoina/toml to decode toml, which fails if a multi-line string has a double quote inside any line, like the following
https://github.com/threefoldtech/jumpscaleX/blob/ea5e7530beae00e8bab63322186ca670eee55fb9/Jumpscale/builder/db/templates/postgres_startup.toml#L5-L14
Will try to open an issue there.

Working flist (based on ubuntu)
https://hub.grid.tf/abdelrahman/psql9.6-ubuntu18.04.flist.md

Waiting for #603 to be solved

Verified on
branch: development_builder
commit: e2cd62f83b2aba6f565cccdd445e3de3bb6a72d5
Now (build, install, start, stop, test and sandbox) are working fine.


Clear command box

Clears the command input box every time a user hits the "enter" button. This minor feature simulates the a CLI (Command Line Interface).

Note: Removal of test cases will be part of the major refactoring to be done for the application. This will be done in a separate PR.


Create components for Workspace Downgrade

Part of Add Workspace Plan Downgrades to NewDot
Doc section
Steps to complete:

Create WorkspaceDowngradePage
Create generic confirmation component
Create generic downgrade component

@carlosmiceli Could we divide this issue into multiple PRs?

@carlosmiceli Could you help to give a more explanation about the difference between these tasks?

Create generic downgrade component
Create WorkspaceDowngradePage

I notice that we need to create these tasks:

Create Plan Type Page
Create a generic success component
Upgrade #admins messaging and workspace owner whisper

Sorry about the delay, have been sick the last couple of days 🙏
I don't see why not have multiple PRs if you think that's best.

@DylanDylann isn't the success component the same as the confirmation component?
When you say plan type page, do you mean the workspace downgrade page? Btw, I think @jayeshmangwani already put that page together on the other PR, but since it's a PR about routes I think it shouldn't be included there. Can we move that into a the new PR that Dylan is going to work on so it's better organized according to the issues' scopes?
I'll create an issue about upgrades soon, but let me get back to you on that later this week 👍

@carlosmiceli

This is the plan type page

This is the workspace downgrade page

This is the confirm/success page (confirm page and success page are same 😄 )

Handle downgrade message in admin room

@carlosmiceli And in this issue, we will implement:

The plan type page
The downgrade page
The confirm/success page

Not sure if it's helpful or not, but the downgrade page and confirm/success page follow the same pattern/layout as our upgrade equivalent of those screens. So ideally we could reuse a lot of styles from those upgrade screens that are already built.

@dannymcclain Thanks Danny, good idea!
@DylanDylann Yes, let's handle those three elements on this issue. We need to move the downgrade page from this PR into your PR(s) though, so it's better organized. cc @jayeshmangwani

Sure 🚀

@DylanDylann thanks for checking me on the issue description, I just updated it according to this section and images in the doc, I think this will be clearer: https://docs.google.com/document/d/1GDPmbvV47VVxucr-tHYjH80VylcJptP0RfGMVy3zanY/edit?tab=t.0#bookmark=id.mnt3pa1wzv
It's 5 items now (do it in as many PRs as you'd like), I'll create another issue for the messaging portion.

I am woking on PR, try to open PR on Friday

Great, thanks sir! I'll be ready to look at it today or during the weekend, let me know if you need anything :)

Let's me know when the Downgrade API is done. Thanks

@DylanDylann , have you created a page for the plan type? If so, please raise a PR so I can review it.
cc: @carlosmiceli

This requires a BE update before we can implement the App PR

You can raise a FE changes PR so I can use that page for the locked type. Please go ahead with the FE changes, and we can implement the BE later. What do you think, @carlosmiceli ?

It required another PR to integrate API. Please create a PR for https://github.com/Expensify/App/issues/51768 and include the plan type on your PR.
When the API is done, I will raise PR for this issue and https://github.com/Expensify/App/issues/51764

@jayeshmangwani Let's make it flexible to process the project asap

Sounds good! I'll implement this in the PR today.

Let's make it flexible to process the project asap

@DylanDylann sorry I didn't get this, I have blocked you in any way

@DylanDylann You mentioned you were going to implement this page, so I should wait for you, right?

Small update, I just realized that the copy in the Figma doesn't match the copy we agreed on in the doc for the Plan type RHP. Can we update to:

Choose a plan that's right for you. For a detailed list of features and pricing, check out our plan types and pricing help page.

Additionally, we slightly updated the URL to the pricing help page to https://help.expensify.com/articles/new-expensify/billing-and-subscriptions/Plan-types-and-pricing

Can we update to:
Choose a plan that's right for you. For a detailed list of features and pricing, check out our plan types and pricing help page.

Sure, will update this copy and URL

@carlosmiceli Have we completed API for Workspace downgrade?

Hey @DylanDylann sorry for being a bit delayed here, there are no updates yet because I'm tied down with a couple of critical issues. I'd definitely suggest to get the FE work done as advanced as possible before needing the API requests to work fully, so we can get a bit ahead. Do you need everything ready from the BE to make progress here?

@carlosmiceli Any update on the BE changes

It's under review, hopefully merged soon! 🙏

Final BE PR has been merged :)


Namespaced Models

Hi,
I am using devise_token_auth with namespace models. My model is V1::Party. Even though authentication is working fine, the authenticate method is authenticate_v1_v1_party!. I suppose it should have been authenticate_v1_party!
I had modified the migration and model for namespaces
create_table :v1_parties, id: :uuid do |t|
# Snipped
end

module V1
  class Party < ActiveRecord::Base
  rolify :role_cname => 'V1::Role'
    # Include default devise modules.
    devise :database_authenticatable, :registerable,
      :recoverable, :rememberable, :trackable, :validatable,
      :confirmable, :omniauthable
    include DeviseTokenAuth::Concerns::User
  end
end

Any suggestions?
Thanks,
V

Hi there @kaevee    ,
In an effort to cleanup this project and prioritize a bit, we're marking issues that haven't had any activity in a while with a "close-in-7-days" label. If we don't hear from you in about a week, we'll be closing this issue. Obviously feel free to re-open it at any time if it's the right time or this was done in error!
If you are still having the issue (especially if it's a bug report) please refer to our new Issue Template to provide some more details to help us solve it.
Hope all is well.


https://nix.fuel.network/ doesn't work

the link in here https://github.com/FuelLabs/fuel.nix doesn't work

Thanks for the report 👌

It is now operational.


[Hotfix] Use the 1es-windows-2022 image instead of 2019 for secret validation

This should fix https://github.com/dotnet/dnceng/issues/740
I Validated that this fixes the stage by running this build https://dev.azure.com/dnceng/internal/_build/results?buildId=2256263&view=results which shows the different outcomes of running in the 2022 image versus the 2019 image.
(Main builds are also broken, but I'm focusing on the rollout right now, and I'll let this merge back to fix main)

(I don't have a root cause as of why this fixes things. If someone thinks it's important to get to the bottom of this one, we can RCA the issue for someone to dig)


Intron insertion bug? (needs resolving)

In some cases, the intron insertion sites might be close enough that they'll be inserted right next to each other. Make sure they aren't overlapping in a weird way.

Have resolved the issue, which turned out to be due to introns being inserted sequentially at non-unique insertion sites.


Fixes for debugging Web Tests

Debugging web tests was broken, simple fix

Codecov Report

Merging #10503 (c3bae02) into main (b821681) will decrease coverage by 0%.
The diff coverage is n/a.

@@          Coverage Diff           @@
##            main   #10503   +/-   ##
======================================
- Coverage     71%      70%   -1%
======================================
  Files        467      467
  Lines      28073    28073
  Branches    4736     4736
======================================
- Hits       19944    19927   -17
- Misses      6239     6258   +19
+ Partials    1890     1888    -2

Impacted Files
Coverage Δ

src/kernels/execution/cellExecution.ts
71% <0%> (-9%)
:arrow_down:

src/kernels/execution/cellExecutionQueue.ts
81% <0%> (-2%)
:arrow_down:

src/kernels/variables/kernelVariables.ts
56% <0%> (-2%)
:arrow_down:

...ctive-window/editor-integration/codeLensFactory.ts
88% <0%> (-1%)
:arrow_down:

src/notebooks/outputs/tracebackFormatter.ts
100% <0%> (+6%)
:arrow_up:


feat: trigger crate

Built on top of the draft PR #415 and discussed in issue #402
Notable difference: added a type TriggerExtra that encapsulate Router for HttpTrigger or subscriptions for RedisTrigger
@lann @radu-matei

@Mossaka: Reviewing this PR inspired me to make a bunch of changes I've been thinking of doing anyway, one benefit of which being that I was able to simplify some of the new trigger crate code. Take a look: https://github.com/fermyon/spin/pull/418/files/2eaa0c48c09a2f16a4211a169dd047bc2fefc7c1..4b54f9cc1883f3b1e2bff0b2c438a1ad26f1b667

@lann I really like your approach that removes the dependency on Application

I will have an update to resolve the conflicts

I had a commit https://github.com/fermyon/spin/pull/418/commits/3822f5829dae7e1b7b372199ca7e296b38930b95 that changes some struct's names
mostly Runtime -> Execution

I am going to resolve conflicts and merge shortly.


Error during smk-pipeline: "fibertools: error: unrecognized arguments: --nuc-size-cutoff 85"

Hi @mrvollger,
I've faced with a problem during running-smk the pipeline:
Error in rule nucleosome:
    jobid: 257
    input: temp/m64323e_220412_060446_resdir/ft.43-of-80.bam, temp/m64323e_220412_060446_resdir/hmm_model.json
    output: temp/m64323e_220412_060446_resdir/nuc.43-of-80.bam
    log: logs/m64323e_220412_060446_resdir/nucleosome/43-of-80.log (check log file(s) for error details)
    conda-env: fiberseq-smk
    shell:

        fibertools -t 4 add-nucleosomes                 -m temp/m64323e_220412_060446_resdir/hmm_model.json --nuc-size-cutoff 85                 -i temp/m64323e_220412_060446_resdir/ft.43-of-80.bam -o temp/m64323e_220412_060446_resdir/nuc.43-of-80.bam                 2> logs/m64323e_220412_060446_resdir/nucleosome/43-of-80.log

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: 63493901
Logfile logs/m64323e_220412_060446_resdir/nucleosome/43-of-80.log:
=============================================================================================================================================================================================================================================================
/home/nshaikhutdinov/miniconda3/envs/fiberseq-smk/lib/python3.10/site-packages/xgboost/compat.py:36: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
  from pandas import MultiIndex, Int64Index
usage: fibertools [-h] [-t THREADS] [-v] [-V]
                  {bam2bed,add-m6a,add-nucleosomes,model,split,trackhub,bed2d4}
                  ...
fibertools: error: unrecognized arguments: --nuc-size-cutoff 85
=============================================================================================================================================================================================================================================================

Error executing rule nucleosome on cluster (jobid: 257, external: 63493901, jobscript: /net/seq/pacbio/test_tmp/fiberseq/more27_bam_files/.snakemake/tmp.7bkajxy6/snakejob.nucleosome.257.sh). For error details see the cluster log and the log files of the involved rule(s).

I am still trying to figure out what happened; smk pipeline worked without errors during the last few days. Could it be some version discrepancy between xgboost and pandas?

Oh, thank you a lot, Mitchell! Sure!


'PARCEL_WORKERS' is not recognized as an internal or external command, operable program or batch file.

I'm trying to follow the Quick Start steps, but when I run npm run develop I get the error mentioned in the title.

Seems like you'll want to file an issue on gatsby, not npm.

@ljharb I just realized I think it's because I'm on Windows and the commands that get run need to be formatted differently, like set PARCEL_WORKERS=0; gatsby.  But now the command is just ending without seeming to run the local server:

you're probably right, but either way you'll need to close this and file an issue on gatsby instead :-)


Document handling of data submission files with the same name

Question from @JohnMarkOckerbloom :

What happens if a contributor attempts to upload a file to a stream where there's already a file in that stream with the same name?  Is it rejected, or renamed, or does it overwrite the old file?  How can the contributor tell?  Does it make a difference whether the file has been processed yet, and/or the processing has succeeded or failed?

file names are just labels, no replacement, overwriting or rejection happens.

Documented here: https://github.com/pod4lib/aggregator/wiki/Streams-and-their-files#full-dumps-and-incremental-updates


⚠ pishem-otzivi has degraded performance

In 2f06630, pishem-otzivi (https://пишем-отзывы.рф/) experienced degraded performance:

HTTP code: 200
Response time: 7256 ms

Resolved: pishem-otzivi performance has improved in af537a1.


Update entity-remove script to use Rasa

@EmmaWightman Thanks for making it possible to merge! This makes it way easier to add code snippets and other things in the future :)
Just a little thing for the future: Usually, the person who created the PR (me in this case) pushes the Merge button, once all the reviewers (you) agreed. It doesn't matter in this case, and I also only learned this recently. Just so you don't surprise others ;)


Multiple Choice in the Number Matching Prompt

When we were testing this, my colleague and I both got slightly different experiences.
My experience was as documented here, in that the notification had a text box and I had to type in the number.
But my colleague got a multiple choice style prompt. the same as this article I found online:
https://janbakker.tech/number-matching-with-microsoft-authenticator-app-in-azure-mfa/
I prefer the multiple-choice experience that my colleague got, but I cannot for the life of me figure out what he did differently, and the MS Docs don't seem to cover this at all.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ee6ac70d-5853-6a3a-0964-23b46910b4c4
Version Independent ID: a5beda87-9845-8807-6221-7fdc64079587
Content: Passwordless sign-in with Microsoft Authenticator - Azure Active Directory - Microsoft Entra
Content Source: articles/active-directory/authentication/howto-authentication-passwordless-phone.md
Service: active-directory
Sub-service: authentication
GitHub Login: @Justinha
Microsoft Alias: justinha

@danielharkerdp
Thanks for your feedback! We will investigate and update as appropriate.

@danielharkerdp
Apologies for the delayed response. I have assigned the issue to author. @Justinha please review.

@danielharkerdp The 3 number buttons version, that the colleague saw, is called 'tap and select'.
This older system is vulnerable to people just mashing one of the 3 buttons (accidentally the right one) if an attacker sends lots of notifications ("I just want to stop the notifications 😨").
'Number matching' with the keyboard is the modern system, and is recently enabled for all accounts. It also allows for longer numbers in the future. You need to have an updated Microsoft Authenticator app to get this new modern freely enter numbers prompt. Maybe the colleague had updates disabled for the app.
Administrators can set the authentication methods policy per Azure AD user, to switch to the old slightly more vulnerable 3 button 'tap and select' version.
I think this ticket can be closed.

#please-close


Added support for opening notes in new windows. Fixes #69

@klauscfhq I went ahead and attempted to add this feature since it's something I could use when on Ubuntu. This is my first time working with Electron, so comments and suggestions are requested.
I read/scanned through the documentation, then gave it a try. I may have missed something that I'm unaware of with window management (see Questions below), but my tests looked promising. They behave as expected for macOS close/quit behavior, respecting mainWindow as the main window.
Let me know what changes you would like to see before considering a merge and I'll be happy to make them 👍

Tested on: macOS 10.13.4 and Ubuntu 18.04. (I don't currently have a Windows environment set up to test this on).

Added "New Window" button to top of note frames. Decided against adding to the notes sidebar because that would require adding several elements with event listeners to the DOM.
Changed menu.js actions to affect the focused window.
Refactored createMainWIndow() into New, Main, and Sub variations.
New windows open at (50, -50) offset from the window that spawned it.
New windows open with a width that approximates the mainWindow note frame (subtracting the sidebar widths).
New windows auto hide the sidebar to be consistent with the native desktop apps.
Went with show: true immediately on new windows because waiting for them to be ready still produced a momentary blank window and the lag time from click to open could cause people to click multiple times.

Fix

Changed ipcMain.on('export-as-pdf') to get title from noteWindow rather than global mainWindow

Potential Changes/Additions

Set window title to note title. This would be better usability in OS window lists.
Add a draggable region when sidebar is hidden on macOS. Currently have to exit focus mode to move the note window. This problem doesn't exist on Ubuntu where the window chrome is present.
Use an icon instead of the "New Window" button

Omissions

Toggling the theme only changes the focused window. Any new windows inherit the last set theme. This could be seen as a feature, allowing the user to choose a theme for each note window. The alternative, if I'm not mistaken, would be to loop through all windows during a toggle.

Questions

When a new window is opened, the spawning window is sent two close events with wasClean: true, but then followed by Unrecoverable close condition, not retrying. This doesn't seem to affect the functionality, but worth bringing up. Any suggestions on this?

Thanks for taking the time to review, @klauscfhq. I'll make the changes this weekend or early next week and update the PR here. :+1:

Can't wait for this feature! Is it going to be merged?

What is the status of this feature? Has anything changed within the last 3 years? Unfortunately, this missing feature is a show stopper for me... Any chance that it will be implemented/merged?


Role change not synced with clients for spymasters

role isn't synced properly for players who were spymasters in the last turn on the website.

fixed in: https://github.com/voldyman/codenames.plus/commit/a99f7b0e85d5a340525f28c92dd922c59d3b61b7


Add onExecutionSetLocalVariable to CallActivityMock

In one of our tests we needed this and had to program the feature ourselves. If would be cool if the lib had it.

If you already have an impl, could you provide a PR? Would certainly speed up things (and help me to better understand).

Hrm... I must dig out the project. I even forgot which one it was. I'll try.


Export and import command problems

No error message for entering just the word export.
The message 3 contacts have been imported into your AddressBook! is displayed even though the duplicated persons are found.

issue fixed


Not expected ZF2 session locking during AJAX calls

The problem is that I'm making multiple ajax calls to a specific action on my system, these calls are making outer calls to a distant system and are waiting significant amount of time to complete and return results. On my system I am using ZF2 AuthenticationService for user authentication (saving the identity to the session). I browse trough the code and found out the different approach that ZF2 use to work with the session data,  I so how the SessionManager each time write into the $_SESSION without caring does this session data is changed or not. This writing into the session global makes the session locks. In my ajax requests I need to read the data from the session but when I try to getIdentity (access the session data) the Session locks and other ajax request that needs to run asynchronously stops to wait until the Seesion unlocks. Does ZF2 has a capability to workarround this issue? How? Thank's a lot

Is this still an issue?


feat: Export command - Export SBOM from storage

Export SBOMs Command
Description
Suggestion for a Save Command, Considering @jhoward-lm Comment.
Typically, users populate the database using the fetch command. This new command will allow users to save SBOMs to their local filesystem.
Usage suggestion.
Save SBOM file(s) from Storage to Filesystem

Usage:
  bomctl save [flags] SBOM_ID...

Flags:
  -e, --encoding string   the output encoding [spdx: [text, json] cyclonedx: [json] (default "json")
  -f, --format string     output format [spdx, spdx-2.3, cyclonedx, cyclonedx-1.0, cyclonedx-1.1, cyclonedx-1.2, cyclonedx-1.3, cyclonedx-1.4, cyclonedx-1.5]
  -h, --help              help for save

Issues

Handling multiple SBOM IDs with a single output file:
Option 1: Output as tar archive
Option 2: Append index to file path
Option 3: Do not support multiple SBOM IDs in the command
Open to other suggestions

Type of change

[ ] New feature (non-breaking change which adds functionality)
[ ] This change requires a documentation update

How Has This Been Tested?
Waiting for feedback before implementing tests.

After giving it some thought, I think it makes more sense to me for this sub command to be called export, where if no positional parameters are provided it should write to stdout for piping to other tools. I still think the fetch option should be called --save however. Thoughts?
The reason is for parity with a potential import command I'm kicking around, which would basically just be fetch except designed for local files and stdin input.

If the fetch command allows fetching of externalRefs, I think a save command that could put "tree structures" as sbom files.

What are your thoughts for the naming convention of the files in that tree structure, or ways to specify multiple output file names and map them to the input URLs

If the import command allows a single SBOM to be piped in, I would expect an export to also pipeline a single SBOM.

It might not necessarily have to be a single input or output SBOM, thinking along the lines of how tools like xargs handle these situations. Input or output could basically be a whitespace delimited series of multiple super long SBOM content strings

In my opinion, large stdout and stdin strings are useful when using pipes but not very practical when printed directly to the terminal.
I also agree that the save command should be renamed to export. IMHO one if the use case is exporting an SBOM to a third-party system or generating a report. In such cases, outputting a collection/tree of SBOMs files may be useful and more natural to some systems.
I was under the impression that creating a unified SBOM should be the responsibility of the merge command, not the export/save command.

Sorry for the delay, i was yet on another small vacation.

After giving it some thought, I think it makes more sense to me for this sub command to be called export, where if no positional parameters are provided it should write to stdout for piping to other tools. I still think the fetch option should be called --save however. Thoughts?

Changed save to export command as suggested.

It might not necessarily have to be a single input or output SBOM, thinking along the lines of how tools like xargs handle these situations. Piped I nput or output could basically be a whitespace delimited series of multiple super long SBOM content strings

If the import command allows a single SBOM to be piped in, I would expect an export to also pipeline a single SBOM.

I Patched the code with both these suggestions, what do you think?

When uses output-file option only one SBOM id is allowed for command.
When not using output-file option all Sboms are pushed out to stdout.

@jhoward-lm @idunbarh

@idunbarh @jhoward-lm I think i fixed all your comments.
Have another look at it.
I ran a local test and it worked as expected.
go run main.go fetch https://raw.githubusercontent.com/protobom/protobom/main/test/conformance/testdata/cyclonedx/1.5/json/bom-1.5.json --cache-dir .tmp

go run main.go export urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79 --cache-dir .tmp

Current Status of usage.
Export SBOM file(s) from Storage

Usage:
  bomctl export [flags] SBOM_URL...

Flags:
  -e, --encoding STRING    the output encoding [spdx: [text, json] cyclonedx: [json] (default json)
  -f, --format STRING      Output Format:
                           	- spdx,
                           	- spdx-2.3,
                           	- cyclonedx,
                           	- cyclonedx-1.0,cyclonedx-1.1,
                           	- cyclonedx-1.2,
                           	- cyclonedx-1.3,
                           	- cyclonedx-1.4,
                           	- cyclonedx-1.5
                           	 (default cyclonedx-1.5)
  -h, --help               help for export
  -o, --output-file FILE   Path to output file

Global Flags:
      --cache-dir string   cache directory [defaults:
                           	Unix:    $HOME/.cache/bomctl
                           	Darwin:  $HOME/Library/Caches/bomctl
                           	Windows: %LocalAppData%\bomctl]
      --config string      config file [defaults:
                           	Unix:    $HOME/.config/bomctl/bomctl.yaml
                           	Darwin:  $HOME/Library/Application Support/bomctl/bomctl.yml
                           	Windows: %AppData%\bomctl\bomctl.yml]
  -v, --verbose            Enable debug output

@jhoward-lm @idunbarh
I decided to set the default format string to cyclonedx-1.5.

Should we consider changing the default to something else?
Do we retain the original document format somewhere that the command can use?

This might be related to https://github.com/protobom/protobom/issues/213.

Also for the last linting issue I think changing the linting rules to allow the max cyclomatic complexity to 13 is reasonable (or take an exception and suppress error).

I think the complexity rule is there for a reason to keep our code easy to read and understand, so I would prefer to avoid modifying the golangci-lint configs if at all possible.
The fix should be pretty simple, just create a private function for the first switch statement, something like
func getFormatMap(encoding string) (map[string]formats.Format, error) {
	switch encoding {
	case formats.JSON:
		return JSONFormatMap(), nil
	case formats.TEXT:
		return TVFormatMap(), nil
	case formats.XML:
		return XMLFormatMap(), nil
	default:
		return formats.EmptyFormat,
			errUnknownEncoding
	}
}

@idunbarh @jhoward-lm Squashed offending commit, my bad.


[gpu] [python] LightGBMError: No OpenCL device found

Description

Reproducible example
Connect to localhost:8888 jupyter notebook
from lightgbm import LGBMClassifier
from sklearn.datasets import make_moons
model = LGBMClassifier(boosting_type='gbdt', num_leaves=31, max_depth=- 1, learning_rate=0.1, n_estimators=300, device = "gpu")
train, label = make_moons(n_samples=300000, shuffle=True, noise=0.3, random_state=None)
model.fit(train, label)

Results in
LightGBMError                             Traceback (most recent call last)
<ipython-input-1-3cadc7bec646> in <module>
      3 model = LGBMClassifier(boosting_type='gbdt', num_leaves=31, max_depth=- 1, learning_rate=0.1, n_estimators=300, device = "gpu")
      4 train, label = make_moons(n_samples=300000, shuffle=True, noise=0.3, random_state=None)
----> 5 model.fit(train, label)

Environment info
LightGBM version or commit hash: Docker version
Command(s) you used to install LightGBM
Run everything according to https://github.com/microsoft/LightGBM/tree/master/docker/gpu
mkdir lightgbm-docker
cd lightgbm-docker
wget https://raw.githubusercontent.com/Microsoft/LightGBM/master/docker/gpu/dockerfile.gpu
docker build -f dockerfile.gpu -t lightgbm-gpu .
nvidia-docker run --rm -d --name lightgbm-gpu -p 8888:8888 -v /home:/home lightgbm-gpu

Full traceback
---------------------------------------------------------------------------
LightGBMError                             Traceback (most recent call last)
<ipython-input-1-3cadc7bec646> in <module>
      3 model = LGBMClassifier(boosting_type='gbdt', num_leaves=31, max_depth=- 1, learning_rate=0.1, n_estimators=300, device = "gpu")
      4 train, label = make_moons(n_samples=300000, shuffle=True, noise=0.3, random_state=None)
----> 5 model.fit(train, label)

/opt/conda/envs/py3/lib/python3.8/site-packages/lightgbm/sklearn.py in fit(self, X, y, sample_weight, init_score, eval_set, eval_names, eval_sample_weight, eval_class_weight, eval_init_score, eval_metric, early_stopping_rounds, verbose, feature_name, categorical_feature, callbacks, init_model)
    888                     valid_sets[i] = (valid_x, self._le.transform(valid_y))
    889
--> 890         super().fit(X, _y, sample_weight=sample_weight, init_score=init_score, eval_set=valid_sets,
    891                     eval_names=eval_names, eval_sample_weight=eval_sample_weight,
    892                     eval_class_weight=eval_class_weight, eval_init_score=eval_init_score,

/opt/conda/envs/py3/lib/python3.8/site-packages/lightgbm/sklearn.py in fit(self, X, y, sample_weight, init_score, group, eval_set, eval_names, eval_sample_weight, eval_class_weight, eval_init_score, eval_group, eval_metric, early_stopping_rounds, verbose, feature_name, categorical_feature, callbacks, init_model)
    681             init_model = init_model.booster_
    682
--> 683         self._Booster = train(params, train_set,
    684                               self.n_estimators, valid_sets=valid_sets, valid_names=eval_names,
    685                               early_stopping_rounds=early_stopping_rounds,

/opt/conda/envs/py3/lib/python3.8/site-packages/lightgbm/engine.py in train(params, train_set, num_boost_round, valid_sets, valid_names, fobj, feval, init_model, feature_name, categorical_feature, early_stopping_rounds, evals_result, verbose_eval, learning_rates, keep_training_booster, callbacks)
    226     # construct booster
    227     try:
--> 228         booster = Booster(params=params, train_set=train_set)
    229         if is_valid_contain_train:
    230             booster.set_train_data_name(train_data_name)

/opt/conda/envs/py3/lib/python3.8/site-packages/lightgbm/basic.py in __init__(self, params, train_set, model_file, model_str, silent)
   2232             params_str = param_dict_to_str(params)
   2233             self.handle = ctypes.c_void_p()
-> 2234             _safe_call(_LIB.LGBM_BoosterCreate(
   2235                 train_set.handle,
   2236                 c_str(params_str),

/opt/conda/envs/py3/lib/python3.8/site-packages/lightgbm/basic.py in _safe_call(ret)
    108     """
    109     if ret != 0:
--> 110         raise LightGBMError(_LIB.LGBM_GetLastError().decode('utf-8'))
    111
    112

LightGBMError: No OpenCL device found

Additional Comments
I have seen similar issues that are closed as resolved. As I understand the solution was to add
mkdir -p /etc/OpenCL/vendors &&  echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd. But this now is included into the Dockerfile.

@aidiss @valentasgruzauskas Are you both using WSL?

@StrikerRUS no, I am on linux

@aidiss

I have seen similar issues that are closed as resolved. As I understand the solution was to add
mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd. But this now is included into the Dockerfile.

I was encountering the same issue as you and was looking at these same solutions.
But it turns out that for me libnvidia-opencl.so.1 is no longer present anywhere within the filesystem when running the Dockerfile at https://github.com/microsoft/LightGBM/blob/master/docker/gpu/dockerfile.gpu for some reason. Installing nvidia-opencl-icd-375 adds it in. Yet running clinfo it still shows 0 devices are present.

I have the same issue, I'm using WSL.

I haven't tried running sample code since my last comment except just now, but running it now in the latest commit it doesn't give this error anymore. Haven't checked why/how though!

It was working good for few months.
Now it started receiving same error.


Pass EventTypeAutoCreate enabled flag to the data plane

Fixes #3220

Proposed Changes

Add the feature flag to the Ingress contract
In each reconcile loop where we set the Ingress contract, set the feature flag

/cc @creydr @Vishal-Chdhry

/cc @Leo6Leo

https://github.com/Leo6Leo/eventing-kafka-broker/blob/32dfed583ceb02c4fa1581b3bffab07d6ec0e7d6/data-plane/core/src/test/java/dev/knative/eventing/kafka/broker/core/reconciler/impl/ResourcesReconcilerImplTest.java#L436
Should the test here also need to be modified? @Cali0707

https://github.com/Leo6Leo/eventing-kafka-broker/blob/32dfed583ceb02c4fa1581b3bffab07d6ec0e7d6/data-plane/core/src/test/java/dev/knative/eventing/kafka/broker/core/reconciler/impl/ResourcesReconcilerImplTest.java#L436
Should the test here also need to be modified? @Cali0707

@Leo6Leo I don't think so? This PR is just passing the flag into the data plane, not updating the code anywhere to use it.
@pierDipi WDYT?


Error adding segmented controller

I'm trying to add a segmented control using this code:
        // add custom segmented control
        self.cSegControl = XMSegmentedControl(frame:tableHeaderAbsFrame, segmentTitle: ["one", "two", "three"], selectedItemHighlightStyle: XMSelectedItemHighlightStyle.BottomEdge)
        cSegControl.backgroundColor = UIColor(red: 255/255, green: 255/255, blue: 255/255, alpha: 1)
        cSegControl.highlightColor = UIColor(red: 255/255, green: 0/255, blue: 91/255, alpha: 1)
        cSegControl.tint = UIColor.black
        cSegControl.highlightTint = UIColor.black
        cSegControl.delegate = self
        self.view.addSubview(cSegControl)

but I get this error

2017-03-06 17:50:39.664 MyAPP[56793:9977216] -[XMSegmentedControl.XMSegmentedControl segmentPressed:]: unrecognized selector sent to instance 0x7fb9a0d7bc00
2017-03-06 17:50:39.674 MyAPP[56793:9977216] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[XMSegmentedControl.XMSegmentedControl segmentPressed:]: unrecognized selector sent to instance 0x7fb9a0d7bc00'
*** First throw call stack:

not sure why the tap events don't get handled.

Are you implementing the delegate in the view controller that hosts the segmented control?
Sent from my iPhone

On Mar 6, 2017, at 12:55 PM, cdepace<EMAIL_ADDRESS>wrote:
I'm trying to add a segmented control using this code:
    // add custom segmented control
    self.cSegControl = XMSegmentedControl(frame:tableHeaderAbsFrame, segmentTitle: ["one", "two", "three"], selectedItemHighlightStyle: XMSelectedItemHighlightStyle.BottomEdge)
    cSegControl.backgroundColor = UIColor(red: 255/255, green: 255/255, blue: 255/255, alpha: 1)
    cSegControl.highlightColor = UIColor(red: 255/255, green: 0/255, blue: 91/255, alpha: 1)
    cSegControl.tint = UIColor.black
    cSegControl.highlightTint = UIColor.black
    cSegControl.delegate = self
    self.view.addSubview(cSegControl)

but I get this error
2017-03-06 17:50:39.664 MyAPP[56793:9977216] -[XMSegmentedControl.XMSegmentedControl segmentPressed:]: unrecognized selector sent to instance 0x7fb9a0d7bc00
2017-03-06 17:50:39.674 MyAPP[56793:9977216] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[XMSegmentedControl.XMSegmentedControl segmentPressed:]: unrecognized selector sent to instance 0x7fb9a0d7bc00'
*** First throw call stack:
not sure why the tap events don't get handled.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Yes, I solved using copy/pasting the XMSegmentedControl.swift file from the example directly to my project. Integrating the pod was giving this error.

Oh that is weird. I thought that was fixed. Thanks for letting me know!


trealla: 2.53.48 -> 2.55.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/trealla-prolog/trealla/releases.
meta.description for trealla is: Compact, efficient Prolog interpreter written in ANSI C
meta.homepage for trealla is: https://trealla-prolog.github.io/trealla/
meta.changelog for trealla is: https://github.com/trealla-prolog/trealla/releases/tag/v2.55.3
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2.55.3 with grep in /nix/store/p5379kplmq665zrkr50chb06902lxck9-trealla-2.55.3
found 2.55.3 in filename of file in /nix/store/p5379kplmq665zrkr50chb06902lxck9-trealla-2.55.3

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

trealla

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/p5379kplmq665zrkr50chb06902lxck9-trealla-2.55.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A trealla https://github.com/r-ryantm/nixpkgs/archive/d1163a9f12b6a83d43298d9eedf5f8277258de65.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/d1163a9f12b6a83d43298d9eedf5f8277258de65#trealla

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/p5379kplmq665zrkr50chb06902lxck9-trealla-2.55.3
ls -la /nix/store/p5379kplmq665zrkr50chb06902lxck9-trealla-2.55.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

trealla

Maintainer pings
cc @siraben @AndersonTorres for testing.

@merge-bot merge


⚠ Bofrost.de has degraded performance

In 83eee50, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1647 ms

Resolved: Bofrost.de performance has improved in 68b589a after 10 minutes.


New plot is very expensive for frequent data changes.

I would advise using Plotly.react instead of newPlot and relayout.
This will simplify your code and instead of rebuilding a whole new plot it just redraws the styling and traces.
plotly.react docs
I actually wrote a tiny implementation of your bigger idea of embedding vuejs in plotly.js
https://medium.com/@bobthomas295/vuejs-and-plotly-js-reactive-charts-da9b3b59f2dc
Here you can see how easy it is to use the Plotly.react and the speed you get with it.

Fixed in 14b8c07

Sweet

Should the window autoresize listener use react rather than newPlot?

Fixed


Info about dependencies and mechanism

Hello,
Thanks for a interesting project!
I have some question about how jHardware works.

I noticed that JNA is in the Maven dependencies of the project (transitively though jSensors). But at the same time the read-me file says that JNA is not used. So can I use jHardware without having JNA on my classpath?  Can I use some functionality from jHardware without JNA?

I also noticed jPowerShell in the dependencies. Does that mean that PowerShell must be installed on a Windows system for jHardware to work?

Is PowerShell and JNA the tools that jHardware uses to read hardware information? How does it work on Linux? Does it invoke other native commands to get information?

The read-me text says that jHardware is "using pure Java". I don't think this is accurate if you are in fact using JNA and calling native programs such as PowerShell to get information.

I think this information would be useful have in the read-me text!
Thank you!

Thanks for the information! These things are helpful to know.


Ignore incomplete jobs with reason 'quit'

In this case the worker has been restarted within a reasonable
time (systemd had not had to kill it). This usually happens during
deployment or when the worker is restarted/stopped for other
reasons. I suppose we can ignore these incompletes.
Note that more than 50 % of the incomplete jobs on o3 within the
last 24 hours have the reason 'quit'.

ok, before we ignore let's make sure we are on the same page, just to verify:

are jobs incompleting with result quit automatically retriggered?

I've just tried and yes, jobs are duplicated. However, the implementation is a little bit problematic:
[debug] [pid:28438] REST-API call: POST http://localhost:9526/api/v1/jobs/1219/status
^C[info] [pid:28438] Quit due to signal INT
[debug] [pid:28438] Stopping job 1219 from http://localhost:9526: 00001219-opensuse-Tumbleweed-DVD-x86_64-Build20200203-create_hdd_textmode@64bit - reason: quit
[debug] [pid:28438] REST-API call: POST http://localhost:9526/api/v1/jobs/1219/status
[info] [pid:28438] Trying to stop job gracefully by announcing it to command server via http://localhost:20013/qFN_BFK7eJCmMjiT/broadcast
[info] [pid:28438] Isotovideo exit status: 0
[info] [pid:28438] +++ worker notes +++
[info] [pid:28438] End time: 2020-02-07 12:54:19
[info] [pid:28438] Result: quit
[debug] [pid:28438] Job 1219 stopped as incomplete
[debug] [pid:28438] duplicating job 1219
[debug] [pid:28438] REST-API call: POST http://localhost:9526/api/v1/jobs/1219/duplicate?dup_type_auto=1
[warn] [pid:28438] Can't open /hdd/openqa-devel/openqa/pool/1/testresults/result-welcome.json for result upload - likely isotovideo could not be started or failed early. Error message: No such file or directory
[debug] [pid:28438] REST-API call: POST http://localhost:9526/api/v1/jobs/1219/status
[debug] [pid:28634] Uploading artefact isosize-1.txt
[debug] [pid:28438] REST-API call: POST http://localhost:9526/api/v1/jobs/1219/set_done?reason=quit&result=incomplete
[debug] [pid:28438] Job 1219 from http://localhost:9526 finished - reason: quit
[debug] [pid:28438] Informing http://localhost:9526 that we are going offline

That means the duplication is triggered from the worker side. I don't know why it has been implemented like this. I'd prefer having the job duplication triggered by the web UI when the worker marks the job as incomplete. So right now, when the duplication API call fails we can end up with jobs which the reason 'quit' and no duplicate. I've simulated it here by changing "jobs/$job_id/duplicate" to "jobs/$job_id/duplicat" in the worker code. If I'd changed the worker to make an API error out of it we'd be on the safe side without changing too much.

ok, maybe that approach is better, I don't know. However I think ignoring the "quit" here is the right call.


⚠ Redacted has degraded performance

In 9fd19da, Redacted ($RED) experienced degraded performance:

HTTP code: 200
Response time: 4053 ms

Resolved: Redacted performance has improved in 1293c7d.


get rid of legacy docs that will never be useful

I'm asking others to make obvious additional commits to this. Please let them be file deletions only lest changes get lost amid 100s of deletions.

Additional candidates for deletion

library/src/charmory/msw-foods.py
library/src/charmory/experimental/feature.py
library/src/charmory/experimental/food_track.py

Also, are you keeping docs/original for reference/future use? Or can we just drop that whole folder?


Multiplayer

Relates to https://github.com/spelunky-fyi/overlunky/pull/254

Merged to main manually...


Incremental compilation RFC

High-level strategy for incremental compilation.
cc @rust-lang/compiler

Awesome!
This is about incremental builds for a single-crate, right? If so, it's worth calling that out.
Also, if I'm correct, these caches are not meant to be shared across build machines, right?

@nikomatsakis The summary talks about debug builds specifically, but IIRC we discussed how this would apply to release builds as well? (I.e., a story a bit like parallel codegen units, where you'd be trading incrementality against optimization potential due to passing LLVM smaller units of code)

@larsbergstrom actually, I believe incremental builds across crates can be done relatively easily, though I didn't discuss it. I will add a TODO item to summarize how that would work.
@aturon yes I updated the summary, my mistake.

@larsbergstrom added a brief note about cross-crate dependencies

@larsbergstrom

Also, if I'm correct, these caches are not meant to be shared across build machines, right?

That is correct.

How does member function name (x.foo()) lookup work in this scheme, particularly in the case of autoderef?  Presumably a failed lookup has to create a dependency on something, but it's not clear what exactly that "something" is.

@eefriedman

How does member function name (x.foo()) lookup work in this scheme, particularly in the case of autoderef? Presumably a failed lookup has to create a dependency on something, but it's not clear what exactly that "something" is.

That "something" is the IR tables that indicate what traits are in scope at a given point, as well as those that collect all the impls for a trait (I did not add an exhaustive listing to the RFC). Those will presumably be linked up something like the following:

there will be an edge from the containing module/scope to the tables indicating what traits are in scope, such that if a new use statement is added, portions of those tables are invalidated.
method search will be adding edges from the table of traits to the fns that include method calls.
the coherence pass will add edges from each impl to IR node representing the set of traits of that impl
trait search will add edges from the set of traits for a given impl to the fn using it

That is roughly the idea. Make sense?

Yes, that makes sense; thanks.

As an example of what I think is incremental compilation done right, see MSVC. Not only does it have an incremental compilation + linking mode that works fairly well, but it also has an incremental LTCG mode where it does full link time optimization, just incrementally.

As I understand it, a large benefit of incremental compilation is speed, but there's no mention of tests that  attempt to quantify or ensure that the new world order will be faster. Is there anything more beyond time cargo build?

Not a very helpful comment, but 👍👍👍👍👍

@bstrie

Mind elaborating on what "demand driven style" entails and how it differs from our current approach?

By demand-driven style, what I meant was that we would build a dependency graph that we use to drive compilation. So, for example, we would begin by saying "we need to trans the main function" (assuming an application), so let's try to do that. But to trans the main function, we must know that it is correct, so that would require us to borrow check main. But to borrow check main, we must know it is type correct, so we would first type check it. This in turn would require knowing what its names refer to, so we would run name resolution. During type-checking, we would collect the signature of each fn that gets called would then get explored as well, since we can't type-check main without knowing that. Once we're done with main, we'd do the same procedure for every other fn in the crate. At the end, we'd have translated everything, but we do so depth-first rather than breadth-first. Make sense?

@shepmaster

Since you don't indicate that every path has a unique integer, this seems to imply that you'd have to know if there are any duplicate children before you start naming, or have some amount of mutability to go back and "fix" the first child when you see the second child.

In the actual implementation, every path element also has a disambiguating integer. This begins as zero, but when we create a new def-id, we check if the parent already has a child with that name and, if so, increment the disambiguating integer as many times as we have to until we get a unique name. I can tweak the RFC to reflect the impl more precisely.

@shepmaster

I believe this is an understood limitation, but it may be worth pointing out again that this wouldn't allow bar to be inlined into foo, otherwise you'd end up with two versions of bar.

I don't know what you mean here, actually. Do you mean that if we inlined, then the graph would be wrong? Because that is not the case: this graph refers to the front end's view of things, which is before inlining etc will take place. When we actually do codegen, if foo and bar are placed into the same codegen unit, then yes LLVM may choose to do inlining (and that would be reflected in the dependency graph). I don't think I have a good example graph showing how that would work, but it's described textually in the section on optimization.

@shepmaster

As I understand it, a large benefit of incremental compilation is speed, but there's no mention of tests that attempt to quantify or ensure that the new world order will be faster. Is there anything more beyond time cargo build?

That's right, all the tests I've written are oriented around correctness (and could be passed by just never doing any incremental compilation at all). I do plan on adding unit tests that test the dependency graph: i.e., to have a project where we apply a patch and check what will need to be rebuilt and make sure it matches what we expect. However, those will have to be hand-written. I expect we will continue to monitor compilation time manually, as we do today (though @nrc has made some strides towards putting together something like Are We Fast Yet?). @brson and I were talking about, as part of the transition to MIR, starting to institute some kind of more rigorous compilation-time (and execution-time!) checking, but I don't think we settled on a plan yet.

(thanks @retep998 and @shepmaster for the suggestions and typos, I've incorporated those into the last push)

@nikomatsakis

Do you mean that if we inlined, then the graph would be wrong?

I've no doubt that I'm missing something, so feel free to ignore me as appropriate. However, I'll try to explain what I am seeing.
You said:

But there is no path from the body of bar to the MIR for foo, so changes there need not trigger a rebuild.

In the picture, the arrows ultimately create OBJECT(foo), which in my mind would include MIR(bar) if it were inlined.
Perhaps I'm reading too much into the incomplete graph example presented? In the text you say "only shows the data needed to produce MIR(foo)", but the picture continues on to show LLVM and OBJECT nodes, so my brain was assuming that what was shown was the sum total of what would go in.
If that's the case, perhaps eliding the final two steps from the graph or adding some handwavy extra lines to indicate there are more inputs?
MIR(foo)
 |
 |
 v
LLVM(foo)  <--- Multiple other MIR(*)

With regards to the section "Optimization and codegen units"
We'd be able to do full LTO, instead of only optimizing within codegen units, if we had better integration with LLVM's inlining decisions. Basically LLVM would tell us all the functions that contributed to the inlining decisions for a given function, and we'd store that info. Later on for each function, for each function that was modified, we'd recheck any functions that it contributed to the inlining decisions of. If the inlining decisions do change, then the function is re-optimized. If the inlining decisions don't change, but one of the functions it has inlined was changed, it would still be re-optimized. If the decisions don't change, and none of the inlined functions changed, then it reuses the cached stuff. This is what msvc does and it makes link time optimization silly fast for small changes.

@shepmaster

Perhaps I'm reading too much into the incomplete graph example presented? In the text you say "only shows the data needed to produce MIR(foo)", but the picture continues on to show LLVM and OBJECT nodes, so my brain was assuming that what was shown was the sum total of what would go in.

You are correct that if inlining had happened, there'd have to be additional edges somewhere in the graph to reflect that.

@retep998

We'd be able to do full LTO, instead of only optimizing within codegen units, if we had better integration with LLVM's inlining decisions

This is not sufficient, though, because LLVM may also do interprocedural analysis, such as looking at all the call-sites of a fn and so forth. Now I've not looked in detail at what LLVM does, and it may be that integrating with LLVM more deeply is a good strategy: but, at least initially, I sort of prefer to treat LLVM as a black-box, which is why I wrote the RFC the way that I did. I could be persuaded on this point though.

@nikomatsakis It's just as an eventual goal sort of thing. Get a more conservative incremental compilation working first, but leave room to eventually expand to a much more aggressive incremental LTO.

@retep998 yes, I agree that eventually it would be nice, particularly if LLVM offered this sort of dependency information as a supported API.

It would be interesting to see how could we manage dependencies through trait matching. Maybe just listing all selections will do, but it will force us to re-select everything, and that's already a good percentage of compilation time.
Anyway, the 75% of compilation cost and growing is LLVM time, and another 10% is translation. If we can handle that (not sure about the LLVM part - but translation could probably be managed by serializing MIR + trait selections + ABI. This serialized form could also allow alternate translation backends (e.g. JIT integration if we can't solve LLVM compile-times).

@arielb1

It would be interesting to see how could we manage dependencies through trait matching. Maybe just listing all selections will do, but it will force us to re-select everything, and that's already a good percentage of compilation time.

Can you elaborate here? Are you thinking of trying to cache the results of selections across compilations (presuming the types/impls involved haven't changed)?  Both are interesting questions. I think we can do quite a bit here, though I imagine that initially that we'll (a) not try to cache or reuse the results of selections and (b) structure the depedency graph in a somewhat coarse-grained way, similar to the "impls-sort-by-simplified-self-type" data structure that you put in. That is, we'll have a graph node representing that set, and we'll invalidate it when new impls appear etc. (Though to start we'll maybe make things even simpler, and just have one node per trait.)

@shepmaster

I believe this is an understood limitation, but it may be worth pointing out again that this wouldn't allow bar to be inlined into foo, otherwise you'd end up with two versions of bar.

I tried to add a note to clarify here. HTH.

This looks very good to me. I've added some comments inline. I'm curious what kinds of problems (and solutions) will pop up during implementation.

Few questions:

I'm a bit confused on the stack used for the dependency graph. It seems like we are doing computation as usual and trying to track what is read. This seems wrong to me: I rather build the graph explicitly, and then structure the passes so they can only read a node and it's dependencies. We already have a good idea of when things should depend on interface vs implementation / signature vs body. I'd want to enforce that understanding.

I've long been interested in the "demand-driven style": ideally write some library for memoizing computations over dags and persisting those dags, and then build the compiler around that. Assuming the compiler devs want the "demand-driven style" too, how much does the plan here get us any closer to that goal, vs just delay the inevitable heavy refactor.

@nikomatsakis

Can you elaborate here? Are you thinking of trying to cache the results of selections across compilations (presuming the types/impls involved haven't changed)?

Trait matching for, say, Clone, depends on a good percentage of structs in the program, so every sufficiently large function would have to be recompiled when you change a struct. We may be able to be smarter here.

@michaelwoerister

Why are procedures on a stack? Is this conceptually sound? Are procedure executions really always cleanly nested?

Yeah, I think I will try to avoid it, as I noted later. I can update the RFC to just not mention the stack.

An implementation will also need to take care of canceled compiler invocations, so we don't throw away valid information and have to do a full rebuild in that case.

What do you mean by canceled compiler invocations? i.e., if the process gets killed?

This sounds like we should try to keep and update the old dependency graph. I don't think that's a good idea, since the old graph has undergone some irreversible transformations during persisting and I don't see any benefit in it. Just use the old graph for deleting invalidated cache entries and build a new dependency graph from scratch.

Well, we may implement this by creating a fresh graph, but the important thing is that for those functions that we opt not to recompile, we have to retain the edges from the HIR nodes that did not change, in case they should change in the future. So we are basically stripping out some data from the old graph and then adding fresh data atop, but keeping some core subset.

I'm curious what kinds of problems (and solutions) will pop up during implementation.

Helmuth von Moltke: “No battle plan survives contact with the enemy.” :)

The dependency graph is always acyclic, and thus we have a nice inductive structure to define equality on the obvious way. I wouldn't think of "old" and "new" graphs or "cache invalidation". The cache just needs to remember the input DAGs and the results of pure functions on those dags. Old cache entries may or may not be referenced, but are still valid. [Practically, say one makes a change, and then reverts it?]

@nikomatsakis

What do you mean by canceled compiler invocations? i.e., if the process gets killed?

Yes. Compilation is a potentially long-running process and interrupting it is something users will do. We should be prepared for that. Although, in the beginning it should be enough to just make sure that corrupt data is recognized as such and discarded. Later though, it would be nice not to lose work already done. But that's an implementation issue, I guess.

Well, we may implement this by creating a fresh graph, but the important thing is that for those functions that we opt not to recompile, we have to retain the edges from the HIR nodes that did not change [...]

Ah, I overlooked that the unchanged parts of the graph are not recreated. In that case we have to re-use parts of the old graph, yes. I guess it's not a problem since the graph transformations you suggest to do before persisting seem simple enough and will retain any nodes that actually refer to cache entries. Maybe that's also what @Ericson2314 is trying to tell me in words too smart for me ;)

Talking with @michaelwoerister today, he reminded me of two things the RFC has overlooked:

we have to deal with adjustments to spans as well, particularly for debug info. This is a bit of a pain, because when you change a fn, naturally all the spans for things later in the file will change. The simplest way to address this is to incorporate the filename/line/col into the hash for AST nodes (not the span offset that we use internally). One downside of this is that adding comments will be considered a breaking change. A better solution would be to track spans as an index to an AST node (relative to the start of the fn) so that we can "rebase" them after every compilation.
we need to preserve and redisplay warnings and other non-fatal errors. I am not sure of the best way to do this. The simplest solution might be to save the results and redisplay, but that's tricky because warnings are not tied to an item or anything else, but rather to spans, and they sometimes have multiple parts etc. (That said, we could use the "current in-scope process" to figure out where the warning belongs, I guess.) Given that I think initial impls will try primarily to preserve trans output, the warning issue can probably be put off however.

IIUC 75% of compilation time is spent in LLVM and the high-level objective of this RFC is to run LLVM less often and on less LLVM IR by caching its results, is that right?
If so, why does LLVM take so long? Is there a resource somewhere that explains exactly what the current problem with LLVM is? (doesn't clang, ldc, ... have this problem as well? If not, why not?).  Is it just due to the large amount of LLVM IR rustc feeds it? Or are there other reasons (like the "quality" of the IR)?

The unit of compilation in typical C++ builds is a single source file, and build systems like Make exist to help you ensure ensure that you're only recompiling what's necessary.
The unit of compilation in Rust is an entire crate (equivalent to an entire library or executable in C++), so it's more important for the compiler to figure out what portions of the crate actually need to be recompiled.

@nikomatsakis fortunately adding a comment will only affect the spans for code in that file, so we're not any worse off than C + Make in that regard.

Hear ye, hear ye. This RFC is now entering final comment period.

I think this RFC is good to go. Conceptually it seems sound to me and it contains enough of a concrete outline to start implementing.

I get that as a mere rust user that has never contributed to rustc, I'm basically pontificating on these design decisions that don't affect any public interface. But might somebody comment on whether the alternative of building the dependency graph explicitly and then processing it (lazily or otherwise) as I wrote earlier was considered?

But might somebody comment on whether the alternative of building the
dependency graph explicitly and then processing it (lazily or otherwise) as
I wrote earlier was considered?

Sorry, I meant to reply to your comment earlier. I did consider that design
and I suspect that ultimately we will actually do a bit of both ---
however, I very much want to prevent the dependency graph and the code from
falling out of sync. We have definitely had bad experience in this respect.
Simply building a graph a priori can very easily fall into this trap. If we
do build up a graph up-front, I want to try and refactor the code such that
requesting data where there is no graph edge fails (perhaps by asserting
that the graph edge exists, or by restructuring the API in some way that
it's not even possible).
On Wed, Oct 21, 2015 at 1:33 PM, John Ericson<EMAIL_ADDRESS>wrote:

I get that as a mere rust user that has never contributed to rustc, I'm
basically pontificating on these design decisions that don't affect any
public interface. But might somebody comment on whether the alternative of
building the dependency graph explicitly and then processing it (lazily or
otherwise) as I wrote earlier was considered?
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1298#issuecomment-150016377.

@nikomatsakis Thank you, that is very reassuring. I absolutely agree on the soundness issue, and refactoring to make sure the graph traversals are correct by construction. Sounds like your view is the implicit dependency route is a good way to accurately catch all dependencies without forcing the big refactor, but explicit dependencies is a decent end goal?

I think there will always be some of both. Some dependencies at least
cannot be constructed "up front" but rather must be discovered -- for
example, we have to do method resolution and type-checking to know what
other fns are referenced and hence which dependencies exist.
On Wed, Oct 21, 2015 at 7:33 PM, John Ericson<EMAIL_ADDRESS>wrote:

@nikomatsakis https://github.com/nikomatsakis Thank you, that is very
reassuring. I absolutely agree on the soundness issue, and refactoring to
make sure the graph traversals are correct by construction. Sounds like
your view is the implicit dependency route is a good way to accurately
catch all dependencies without forcing the big refactor, but explicit
dependencies is a decent end goal?
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1298#issuecomment-150088389.

Ah. I envisioned stuff like that working by the traversal of one graph creating another.

To clarify. Suppose we have something like token tree -(macros)-> collection of items -(type-checking and method resolution...)-> collection of MIR -(llvm)-> collection of bitcode.
To really do laziness right with this, not only would the graphs be traversed lazily, but also created lazily. The MIR for each function would be bundled with a thunk to generate the MIR for all referenced functions.
[For any Nix users out there (cough @eddyb cough) this is related to doing things like import (import ./foo.nix).]

Finally, I mentioned earlier I'd love to right some generic library to persist/cache all that. To make that a bit more concrete I was thinking of something like https://github.com/dmbarbour/haskell-vcache or https://github.com/mirage/irmin along with some infrastructure to serialize thunks.

This looks great to me.  The greatest challenge is going to be building a dependency graph that is as precise as possible (to get maximum benefit) without introducing unsoundness.  I don't see any silver bullets here; just "be really careful" and "test a lot".
There may be an interesting class of source code changes affecting lifetime or variance inference where typechecking artifacts are invalidated, but it is theoretically possible to avoid invalidating trans artifacts since lifetimes are erased by then.  I haven't thought of any concrete examples that would be worth exploiting, however.

One thing that is not mentioned in the RFC at all yet is monomorphization and the consequences it has.
The general case of a dependency graph with generic items will look more like the following:
BODY(foo) ----------------------------> TYPECK(foo) ----------------> MIR(foo)
                                          ^ ^ ^ ^                      |
SIG(foo) ----> COLLECT(foo)               | | | |         +------------+------------+
                 |                        | | | |         |            |            |
                 +--> ITEM_TYPE(foo) -----+ | | |         v            v            v
                 +--> PREDICATES(foo) ------+ | |      LLVM(foo'1)  LLVM(foo'2)  LLVM(foo'3)
                                              | |         |            |            |
SIG(bar) ----> COLLECT(bar)                   | |         v            v            v
                 |                            | |    OBJECT(foo'1) OBJECT(foo'2) OBJECT(foo'3)
                 +--> ITEM_TYPE(bar) ---------+ |
                 +--> PREDICATES(bar) ----------+

One complication I can see here is that we can only know after type-checking which monomorphizations are still used, but the proposed algorithm already wants to garbage-collect the on-disk cache right after building the HIR. This has to be accounted for somehow.

True. We don't actually know what monomorphizations we want until trans.
Type-checking doesn't expand things out. I was thinking about
monomorphizations at some point, but I don't remember just what I had in
mind. Regarding GCing of monomorphizations, I think I was originally
thinking that we would just keep all monomorphizations of foo until foo
changed. This does mean though that we might keep some monomorphizations we
no longer need (because they were only being used by bar, and bar
changed). It's also true that the "cache" on disk would have to include the
types in the key, something that the RFC doesn't really discuss explicitly.
I've also been thinking about what it would take to do an early target that
JUST saves LLVM IR and object code. This will require doing a few things
slightly differently, but seems like a good first "spike goal":

We would always recompute the signatures for all items, whether they've
changed or not. This is because
As you point out, we'll have to type-check the bodies for generic fns
that are potentially called, as we may need new monomorphizations thereof.
Probably the easiest way to start would be type-checking all bodies too, or
at least all generic bodies. I think the easiest way to address this would
be by saving and re-loading the MIR, which once it is in use ought not to
be that hard.

On Fri, Oct 23, 2015 at 10:20 AM, Michael Woerister <
<EMAIL_ADDRESS>wrote:

One thing that is not mentioned in the RFC at all yet is monomorphization
and the consequences it has.
The general case of a dependency graph with generic items will look more
like the following:
BODY(foo) ----------------------------> TYPECK(foo) ----------------> MIR(foo)
^ ^ ^ ^                      |
SIG(foo) ----> COLLECT(foo)               | | | |         +------------+------------+
|                        | | | |         |            |            |
+--> ITEM_TYPE(foo) -----+ | | |         v            v            v
+--> PREDICATES(foo) ------+ | |      LLVM(foo'1)  LLVM(foo'2)  LLVM(foo'3)
| |         |            |            |
SIG(bar) ----> COLLECT(bar)                   | |         v            v            v
|                            | |    OBJECT(foo'1) OBJECT(foo'2) OBJECT(foo'3)
+--> ITEM_TYPE(bar) ---------+ |
+--> PREDICATES(bar) ----------+
One complication I can see here is that we can only know after
type-checking which monomorphizations are still used, but the proposed
algorithm already wants to garbage-collect the on-disk cache right after
building the HIR. This has to be accounted for somehow.
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1298#issuecomment-150587949.

We already save the body of monomorphizable fns.

@arielb1

We already save the body of monomorphizable fns.

Yes, but what we are mostly talking about is preserving the monomorphized
LLVM bitcode.
Well, I guess I was saying that for a first draft, it might not be worth
trying to reuse the type-checked body at first. This is because currently
we save the body as part of the metadata in the final end-product, and it
would be work (however little) to save that data somewhere else. Clearly
eventually we want to. I'm mostly just trying to work out what is the
smallest thing we can get working to start.
On Fri, Oct 23, 2015 at 1:18 PM, arielb1<EMAIL_ADDRESS>wrote:

We already save the body of monomorphizable fns.
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1298#issuecomment-150637468.

@nikomatsakis
Maybe convert all translation to use inlining and save the serialized data (we would also need to have some way of stably comparing it for this to work).

@arielb1 I'm not clear on what problem you are proposing to solve here? (I
don't even see that there is a problem that needs solving)
On Fri, Oct 23, 2015 at 4:16 PM, arielb1<EMAIL_ADDRESS>wrote:

@nikomatsakis https://github.com/nikomatsakis
Maybe convert all translation to use inlining and save the serialized data
(we would also need to have some way of stably comparing it for this to
work).
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1298#issuecomment-150679161.

We would always recompute the signatures for all items, whether they've changed or not. This is because
As you point out, we'll have to type-check the bodies for generic fns that are potentially called, as we may need new monomorphizations thereof. Probably the easiest way to start would be type-checking all bodies too, or at least all generic bodies. I think the easiest way to address this would be by saving and re-loading the MIR, which once it is in use ought not to be that hard.

Isn't it proposed anyway that the complete set of items hashed on every compilation?
I think it should not be a problem to just cache object code for starters. Only the dependency graph must be complete and not produce false negatives.

Huzzah! The compiler team has decided to accept this RFC. The expectation is that the actual impl will discover numerous surprises  (we've already found a few) that require adjustments, and that we will come back and update the RFC to be more inline with the final design when that has shaken out a bit.

There's lots of interesting talk about incremental computation in this thread, which is great!
In case anyone was wondering about PL research literature on this topic, these researchers have also been thinking about incremental, demand-driven compilation / computation:

A sound and optimal build system with dynamic dependencies (OOPSLA 2015):  http://www.informatik.uni-marburg.de/~seba/publications/pluto-incremental-build.pdf

Adapton: Composable, Demand-Driven Incremental Computation (PLDI 2014):

http://www.cs.umd.edu/~hammer/adapton/
http://adapton.org

The first paper is more recent, and specialized to a situation similar to the one described in the discussion above (incremental compilation, using demand-driven, dynamic dependency graphs).  The second paper gives a general approach for such incremental, demand-driven computations.  There is follow-on work on adapton.org.

Any update on the state of implementation?

@White-Oak Creation of a dependency graph is being done in rust-lang/rust#30532


Uninstalling a custom genome

Couldn't find it in a search. I'd like to uninstall a custom genome. I'd imagine deleting the directory will not be enough since it is referenced in various loc files throughout the program etc.
Is there an easy way to cleanly remove a genome? Would it be possible to add that to bcbio_setup_genome?

Hi @eyalbenda,
Just deleting the directories should be good enough, you'll have some detritus in the config files but it won't affect anything.

Cool, thanks!
On Mon, May 6, 2019, 13:35 Rory Kirchner<EMAIL_ADDRESS>wrote:

Hi @eyalbenda https://github.com/eyalbenda,
Just deleting the directories should be good enough, you'll have some
detritus in the config files but it won't affect anything.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bcbio/bcbio-nextgen/issues/2812#issuecomment-489766722,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACQHDYMO2XBJDTAYGGPPKVTPUCJARANCNFSM4HLC5Q3Q
.


MediaNotificationController not connecting when using custom LocalBinder

Version
Media3 1.2.0
More version details
No response
Devices that reproduce the issue
Samsung S22 Ultra running Android 14
Devices that do not reproduce the issue
Pixel 6 running Android 14
Reproducible in the demo app?
No
Reproduction steps

Bind our media service using bindService and use custom LocalBinder inside the service.
onConnect function will not called with MediaNotificationController
Therefore, custom media actions are not displayed in the notification.

Expected result
All Custom actions inside notification should display
Actual result
custom media actions are not displayed in the notification.
Media
any media can be used
Bug Report

[ ] You will email the zip file produced by adb bugreport to<EMAIL_ADDRESS>after filing this issue.

Difficult to say. This is a low-level change that you are doing and AFAIU there can be many things that can go wrong. You are kind of doing an open heart surgery on the media session service implementation :).

onConnect function will not called with MediaNotificationController

I guess you are overriding onBind and then provide your own local binder instead of what the actual implementation does. Not sure how you implemented this, but the original implementation provides an implementation of the IMediaSession.aidl that provides connect. The standard implementation implements this here. When connect is called the session is eventually passed to service.addSession(session) which is where connecting the media notification controller is triggered.
That's in a nutshell what you'd need to do. I, as an aside, would discourage doing so though because this changes implementation details that are not part of an API that we can ensure to be stable. What I mean is that your implementation may break when we change something in the AIDL interface or the internal implementation. When you update to a new MEdia3 version, you may be in troubles as in you would have to investigate again. That's not something we can support you with closely also I'm afraid.

Out of interest: What are you trying to achieve with a LocalBinder?
I'm asking because my understanding is that a local binder would be used by clients that bind from the same process. This would mean that this local binder is only given to such a client. A MediaSessionService/MediaLibraryService is typically also accessed by for instance System UI that isn't running in the same process. Today, System UI uses the platform controller to connect, but this may change and in such a case using a local binder could become problematic in the future when such clients move to Media3.
We also made some changes that make connections from the same process more efficient for local binding:
https://github.com/androidx/media/commit/d7044c7b83895be0d350c9fbd4e18226a56708bd
https://github.com/androidx/media/commit/77bccf0dd1c21f7a9588d05fbea5fcdc9874d607


deploy_path cannot be relative for all tasks and recipes

Deployer version: https://github.com/deployphp/deployer/releases/tag/v7.3.1
Deployment OS: macOS

For example, half of the deploy:release recipe can work with a relative path, but some commands require it to be absolute.
It would be great to have everything support relative paths.
Alternatively, it should be clear that alternative paths are not supported.

What do you mean by relative deploy_path? Usally it is a good idea to specify deploy_path from / or from ~.

Might be related to https://github.com/deployphp/deployer/issues/3689#issue-1900532948
Using relative path or ~ does not work when using become as the ~ will always reference the SSH user and not the home of the become user, at least for rsync.

@antonmedv you are right, there is a key piece of information that I was omitting. We built a CI pipeline based on Deployer and added a localhost('ci') host. We wanted to use a ./.deployer/build/ directory in the project root, to perform all the build tasks in.
As the system or paths might change in which we would run the build commands, we hoped we could just use ./deployer/build as our deploy_path and it would interpret this as relative to the project root for the localhost host.
For remote hosts, we do use absolute paths for the deploy_path.
However, relative paths to the user's home directory, would make sense too. We haven't tested using the ~/ syntax yet, which could be an option. Thank you for pointing that out @boesing. As we don't use become for now, it might work for us.