Unable to host IS-6.1.0 webapps externally

Describe the issue:
When hosting the authenticationendpoint and accountrecoveryendpoint apps on a different server according to the official documentation [1] and trying to login to a service provider an error message is received when loading the login page and a NPE is printed on the log:

SEVERE [http-nio-8080-exec-59] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [default] in context with path [/authenticationendpoint] threw exception
	java.lang.NullPointerException
		at org.wso2.carbon.identity.core.DefaultServiceURLBuilder.fetchPort(DefaultServiceURLBuilder.java:318)
		at org.wso2.carbon.identity.core.DefaultServiceURLBuilder.build(DefaultServiceURLBuilder.java:90)
		at org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthParameterFilter.buildAPIPath(AuthParameterFilter.java:134)
		at org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthParameterFilter.getServletRequestWithParams(AuthParameterFilter.java:114)
		at org.wso2.carbon.identity.application.authentication.endpoint.util.filter.AuthParameterFilter.doFilter(AuthParameterFilter.java:82)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
		at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
		at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
		at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
		at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
		at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
		at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
		at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
		at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
		at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
		at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)
		at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1707)
		at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
		at java.lang.Thread.run(Thread.java:750)

As per the above stacktrace, at [5] the IdentityCoreServiceComponent.getConfigurationContextService() method returns null hence the NPE.
This configuration seems to be loaded from the XML files on <IS_HOME>/repository/conf/axis2 directory on a normal setup, but on this kind of setup the external server should be able to load this configuration into context and avoid the NPE.
How to reproduce:

Install an external Tomcat server.
Deploy the authenticationendpoint and accountrecoveryendpoint apps on the external server by following the steps at [3] & [4]. Kindly notice that this scripts are outdated and there are other dependencies that needed to be copied manually to prevent java.lang.ClassNotFoundExceptions. Please consider to update this scripts as well.
Configure the IS to point the external server by following the steps at [1].
Configure the authenticationendpoint and accountrecoveryendpoint apps on an external server by following the steps at [1] & [2]. Kindly notice that the 6.1.0 docs [1] only explains the steps to host the authenticationendpoint app externally, for the accountrecoveryendpoint app you need to refer to 5.11.0 docs [2].
Start the IS and Tomcat servers.
Try to login to a service provider configured on IS.

Expected behavior:
Webapps should work without issues regardless of the server where they are hosted on.
Environment information:

Product Version: IS 6.1.0

[1] https://is.docs.wso2.com/en/latest/references/extend/host-authentication-endpoint-on-a-different-server/
[2] https://is.docs.wso2.com/en/5.11.0/develop/hosting-authentication-endpoint-on-a-different-server/
[3] https://github.com/wso2/samples-is/blob/master/host-endpoints-externally/setup-authentication-endpoint.sh
[4] https://github.com/wso2/samples-is/blob/master/host-endpoints-externally/setup-accountrecovery-endpoint.sh
[5] https://github.com/wso2/carbon-identity-framework/blob/v5.25.92/components/identity-core/org.wso2.carbon.identity.core/src/main/java/org/wso2/carbon/identity/core/DefaultServiceURLBuilder.java#L317

Hi Juan,
Please find the U2 entry [1] and ETA below.
Best Case : 2023-04-28
Most Likely : 2023-05-05
Worst Case : 2023-05-12
Since the ETA is shared we are closing the issue. Feel free to re-open the issue if you need further assistance on this.
[1] https://identity-internal-gateway.cloud.wso2.com/t/wso2internal928/umt/update?updateId=6942&type=viewUpdate
Best Regards,
Thisara

The NPE can be avoided by uncommenting the below config in the ./webapps/authenticationendpoint/WEB-INF/web.xml file

AuthenticationRESTEndpointURL
https://localhost:9443/api/identity/auth/v1.1/

Also, you won't need to manually add the missing jar files in order to avoid the class not found exceptions. Run the below script when hosting the authenticationendpoint in the external server.
https://github.com/ImalshaG/samples-is/blob/login-portal-script/host-endpoints-externally/setup-authentication-endpoint.sh
But still there is one jar is missing. Copy the $IS_HOME/lib/runtimes/cxf3/org.wso2.carbon.identity.mgt.endpoint.util-5.25.92.jar and paste into the
./webapps/authenticationendpoint/WEB-INF/lib folder.

For the account recovery portal the script is updated. Hence no need to manually copy the jars.
The configuration mentioned in this doc suggest to configure as below.
identity.server.service.contextURL=https://localhost:9443/services/

But having the /services/ cause an issue. Hence had to remove that.  Ref - https://github.com/wso2/product-is/issues/10315#issuecomment-1271213532
Then, the validate callback url during password recovery is failing due to recent change done for validating the callback url due to a security issue. That logic fails when hosting the recovery portal outside the IS.

https://github.com/wso2/identity-apps/pull/3129

Removing those logic let the recovery portal working smoothly. But as the validate callback url is required, working on implementing an alternative implementation to do the validation.

https://github.com/wso2/identity-apps/pull/3453
https://github.com/wso2/carbon-identity-framework/pull/4268

With all these fixes, still there is an improvement to be made. The email sent for the account/password recovery along with a reset link is not pointing to the remotely hosted recovery portal. It should be fixed along with this effor.


enable cross-account async copy

Fixes #530.
enables cross-account copy for startCopyFromURL if the accounts both exist on the same azurite instance. this is accomplished by issuing a GET request to the provided copySource URL, which would need to include a SAS token in order for cross-account copy to function properly. if the response is OK, then the validation succeeds and the current metadata copy (no content copy) logic is used.

Please fix the CI failure.

Please also update Support Matrix in ReadMe and Change log

@ patrickcarnahan Thanks for updating! Bit ocurripied last days. Will take a look tmr.


local services accessable with/without ncat in public-bridge

I wonder if I have the correct understanding at all;
for  public-bridgemode, I expect the created nomad-pot to be reachable via the <POT.PUPLIC.BRIDGE.IP>:<PORT>. Looking at the code this would be realized via a deamonized ncat, which should be copied to the pot's filesystem during start.
To me it looks that does not work. I can not see the daemonized process nor can I find the binary in the jails root. (the required setting localhost-tunnel is correctly set to yes.
Do I assume correctly, that the services should be reachable via the public bridge? And if not, how can for instance consul check on them, when the IP's a varying
thanks for help!

pot 0.15.6
nomad-pot-driver  0.9.1
freebsd 13.2-RELEASE-p6

I think this issue is more of a Layer8 problem. Therefore I am closing it. Still I would be interested, how I can oversee portmappings or redirects nomad-pot-driver commanded to pf with FreeBSD board tools. I expected a listing in e.g.
pfctl -snat
pfctl - 'pot-nat' -s nat
pfctl -a 'pot-rdr/*' -s rules
pfctl -a 'pot-rdr/*' -sr`

@einsiedlerkrebs Sorry for not being responsive, busy days around here. You are correct that the tooling for anchors in pfctl is a bit annoying.
What you can do is:
pot show

which will list port redirects of known running pots to you (making use of pfctl).
If you want to see pot's anchors (e.g., for redirects) without using pot itself, you can do
pfctl -a 'pot-rdr/*' -s Anchors

(case sensitive).
And then poke into each of them, e.g.:
pfctl -a 'pot-rdr/*' -s Anchors | xargs -n1 pfctl -s nat -a

Hi @grembo,
No problem. I am seeing my issue here more as a support case, not a bug any more. Still I wanted to keep it in order to document a little.
Beside your helpful hints how to read the firewall anchors I also learned, that there is no easy way to circumvent the ncat process running, since redirects are origin from localhost and only in FreeBSD 13.3 pf rules for this local interface can be applied (https://www.freebsd.org/releases/13.3R/relnotes/).
Is this the reason why ncat is used in the first place?
In the minipot setup (having nomad/consul/pot on the same host) the port checks are failing when localhost_tunnel is disabled. There is no way, to provide the services locally without ncat, right?
and BTW:
When ncat process should be found via ps aux one should have either a large screen or small text size in terminal to be able to grep for the process.

Hi @grembo,
No problem. I am seeing my issue here more as a support case, not a bug any more. Still I wanted to keep it in order to document a little. Beside your helpful hints how to read the firewall anchors I also learned, that there is no easy way to circumvent the ncat process running, since redirects are origin from localhost and only in FreeBSD 13.3 pf rules for this local interface can be applied (https://www.freebsd.org/releases/13.3R/relnotes/). Is this the reason why ncat is used in the first place? In the minipot setup (having nomad/consul/pot on the same host) the port checks are failing when localhost_tunnel is disabled. There is no way, to provide the services locally without ncat, right?

We run a cluster without ncat by using custom networking scripts - this is configured in pot.conf by setting POT_EXPORT_PORTS_PF_RULES_HOOK, see https://github.com/bsdpot/pot/blob/master/etc/pot/pot.default.conf#L53-L63
This allows us to have local consul on the host talk to services as well as services talk to each other in a well controlled way. The script I use is not published yet, as it's quite custom for what we do (it consists of creating a reflect jail on the host, which runs pf itself). If you're interested I can share more details (it's a bit on the complex side though).

and BTW:
When ncat process should be found via ps aux one should have either a large screen or small text size in terminal to be able to grep for the process.

ww (as in wide-reallywide) is your friend, see man ps:
ps auxww

@einsiedlerkrebs tagging you again to make sure you won't miss my response above.

Hi Grembo.

ww (as in wide-reallywide) is your friend, see man ps

Yes. That's seems friendly!

We run a cluster without ncat by using custom networking scripts - this is configured in pot.conf by setting POT_EXPORT_PORTS_PF_RULES_HOOK, see https://github.com/bsdpot/pot/blob/master/etc/pot/pot.default.conf#L53-L63

I have seen this hook in the code and considered indeed to use it, but later thought I might come up with a dedicated ip_tunnel patch. Idea was, to have the possibility to link traffic via ncat to one dedicated interface (not all as in localhost_tunnel) only in order to make consul happy and advertise the service in DNS. But of course consul advertises the address it checks and does not get hold of the dynamic assigned IP address, which was my goal here.
On a glance on nomad-pot-driver it looks like Nomad is not notified about the dynamic used IP address pot chooses.
So since this straw seems also like bit of work ahead; I am indeed interested.

If you're interested I can share more details

Of course I would like to archive a least proprietary solution (In a frame of using the hole cloud system on one host ;) ).
Yesterday FreeBSD 13.3 was released and that might be a possibility to avoid a pf running reflector jail. Still I am curious in your solution since I assume it is most direct as you are very into the information that a component holds at one point in time (pot, pf, nomad-pot-driver, nomad, consul.)
Maybe the POT_EXPORT_PORTS_PF_RULES_HOOK and FreeBSD new capabilities of filter on lo0-interface as well might offer a generic approach which is worth to put into appearance.

Another Idea that came up and could solve the issue I have is running consul in a jail with vnet. That should allow the firewall to run the redirects as well.

The Idea above seem to lead into the same issue (of course ?) because the consul server can not be reached from localhost.

@grembo since I didn't had any success yet, would you mind, to share the script you mentioned?

@einsiedlerkrebs Ok, so here we go (might also be interesting to @pizzamig):
This setup scales pretty well, which means:

You can have many nomad client nodes
Each pot in each nomad client node can talk to every other pot running on each node
Communication between pots is always over the announced service endpoint (even on the same host)
No direct communication between pots on the same host (no shortcuts) => watching the service via consul works

Example is from one compute node in the cluster.
Configuration used:
pot.conf:
POT_NETWORK=<IP_ADDRESS>/10
POT_NETMASK=<IP_ADDRESS>
POT_GATEWAY=<IP_ADDRESS>
POT_EXTIF=compute
POT_DNS_IP=<IP_ADDRESS>
POT_TMP=/opt/pot/tmp
POT_EXPORT_PORTS_PF_RULES_HOOK=/path/to/bin/make-pf-rules.sh
POT_ISOLATE_VNET_POTS=true

make-pf-rules.sh:
#!/bin/sh

if [ -e /usr/local/etc/pot/pfrules.conf ]; then
  # shellcheck disable=SC1091
  . /usr/local/etc/pot/pfrules.conf
fi

if [ -z "$APP_IP" ]; then
  1>&2 echo "Error: Please configure APP_IP"
  exit 1
fi
: "${APP_BITMASK:="24"}"

if [ $# -ne 8 ]; then
  1>&2 echo "Usage: $0 POT_EXTIF _bridge _pot_net _pot_gateway _proto_port _host_port _ip _pot_port"
  1>&2 echo "Example: $0 cluster bridge1 <IP_ADDRESS>/10 <IP_ADDRESS> tcp 32732 <IP_ADDRESS> 80"
  exit 1
fi

set -e
# shellcheck disable=SC3040
set -o pipefail

SCRIPT=$(readlink -f "$0")
SCRIPTDIR=$(dirname "$SCRIPT")

if ! jls -j pot-reflect >/dev/null 2>&1; then
	"$SCRIPTDIR/make-reflect-jail.sh" >/dev/null
fi

POT_EXTIF=$1; shift
_bridge=$1; shift
POT_NETWORK=$1; shift
POT_GATEWAY=$1; shift
_proto_port=$1; shift
_host_port=$1; shift
_ip=$1; shift
_pot_port=$1; shift

# custom redirect rules / can contain various ports depending on required payload
echo "rdr on $_bridge proto {tcp, udp} from $_ip to $POT_GATEWAY port 53 -> <IP_ADDRESS>"
echo "rdr on $_bridge proto udp from $_ip to $POT_GATEWAY port 514 -> <IP_ADDRESS>"
echo "rdr pass log on pot-reflect-a proto $_proto_port from <IP_ADDRESS> to $APP_IP port $_host_port -> $_ip port $_pot_port"
echo "rdr pass log on $POT_EXTIF inet proto tcp from $APP_IP/$APP_BITMASK to $APP_IP port $_host_port -> $_ip port $_pot_port"
echo "rdr log on $_bridge proto tcp from $POT_NETWORK to $APP_IP port $_host_port tag potnat -> $_ip port $_pot_port"
echo "nat log on $_bridge proto tcp from $POT_NETWORK to $_ip port $_pot_port tagged potnat -> $APP_IP"

Content of make-reflect-jail.sh:
#!/bin/sh

if [ -e /usr/local/etc/pot/pfrules.conf ]; then
  # shellcheck disable=SC1091
  . /usr/local/etc/pot/pfrules.conf
fi

for var in APP_BITMASK APP_IP CLUSTER_IF COMPUTE_IF NOMAD_IF UNTRUSTED_IF; do
  if [ -z "$(eval echo "\${$var}")" ]; then
    log "Error: Please configure $var"
    exit 1
  fi
done

set -e
# shellcheck disable=SC3040
set -o pipefail

SCRIPT=$(readlink -f "$0")
SCRIPTDIR=$(dirname "$SCRIPT")

fibs=$(sysctl -n net.fibs)

if [ "$fibs" -lt 2 ]; then
  sysctl net.fibs=2 >/dev/null
fi

JAILDIR="/root/reflectjail"

# cleanup existing jail if necessary
if jls -j pot-reflect >/dev/null 2>&1; then
  "$SCRIPTDIR"/destroy-reflect-jail.sh "$APP_IP/$APP_BITMASK" "$NOMAD_IF"
fi

# create jail and config
mkdir -p "$JAILDIR"
tar -C / -cf - bin etc/pf.os lib libexec sbin | tar -xf - -C "$JAILDIR/."
chflags -R noschg "$JAILDIR"
mkdir -p "$JAILDIR/dev"
cat <<EOF >"$JAILDIR/etc/pf.conf"
nat log on pot-reflect-b to $APP_IP -> pot-reflect-b:0
pass
EOF

# start jail
jail -c vnet name=pot-reflect host.hostname=pot-reflect \
  path="$JAILDIR" mount.devfs persist

# build vnet
epair_a=$(ifconfig epair create)
epair_b=$(echo "$epair_a" | sed "s/a$/b/")

ifconfig "$epair_a" inet <IP_ADDRESS>/30 name pot-reflect-a >/dev/null
ifconfig "$epair_b" name pot-reflect-b >/dev/null

ifconfig pot-reflect-b vnet pot-reflect
jexec pot-reflect ifconfig pot-reflect-b <IP_ADDRESS>/30
jexec pot-reflect sysctl net.inet.ip.forwarding=1 >/dev/null
jexec pot-reflect sysctl net.inet.ip.redirect=0 >/dev/null
jexec pot-reflect route add default <IP_ADDRESS> >/dev/null

# tune devfs and start pf
devfs -m "$JAILDIR/dev" rule apply path pf unhide
jexec pot-reflect pfctl -f /etc/pf.conf -eq

# route nomad range
route add "$APP_IP" <IP_ADDRESS> >/dev/null

# create pseudo interface for nomad to pickup
ifconfig lo create \
  inet "$APP_IP/$APP_BITMASK" \
  name "$NOMAD_IF" fib 1 >/dev/null

# add NAT for outbound cluster (no "sort before uniq" on purpose)
pfctl -qa reflect -F all
echo "
nat on $COMPUTE_IF to $APP_IP/$APP_BITMASK -> $APP_IP
nat on $CLUSTER_IF from 10.192/10 to ! $APP_IP/$APP_BITMASK -> $CLUSTER_IF:0
nat on $UNTRUSTED_IF from 10.192/10 to ! $APP_IP/$APP_BITMASK \
  -> $UNTRUSTED_IF:0
pass out quick log to $APP_IP rtable 0
pass from 10.192/10 to 10.192/10
pass from <IP_ADDRESS>/32 to 10.192/10
pass from $APP_IP/32 to 10.192/10
" | uniq | pfctl -qa reflect -f -

Example pfrules.conf:
APP_BITMASK=24
APP_IP=<IP_ADDRESS>
CLUSTER_IF=cluster
COMPUTE_IF=compute
NOMAD_IF=nomad-pseudo
UNTRUSTED_IF=untrusted

NAT/RDR rules in hosts /etc/pf.conf:
nat-anchor reflect
rdr-anchor reflect

#nat-anchor pot-nat
rdr-anchor "pot-rdr/*"
nat-anchor "pot-rdr/*"

(no additional anchors, the ones created by pot by default don't apply)
Args to nomad:
nomad_enable="YES"
nomad_args="-config=/path/to/client.hcl -network-interface=nomad-pseudo"
nomad_user="root"
nomad_group="wheel"

Additional interfaces involved in our setup:
untrusted: public uplink
   ifconfig_em0_name=untrusted
compute: private vlan interface used to serve apps
cluster: private vlan interface used to communicate with nomad/vault/consul cluster
  vlans_untrusted="cluster compute"
  create_args_cluster="vlan 1001"
  ifconfig_cluster="inet <IP_ADDRESS>/24"
  create_args_compute="vlan 1002"
  ifconfig_compute="inet <IP_ADDRESS>/24"

Now, the way this works is:

nomad-pseudo is a local interface in fib 1 (so it's not reachable from fib 0) - it only exists, so that nomad has an interface to look at (it plays no role in the setup beyond that, that's why it's in a different fib).
nomad uses nomad-pseudo's IP address (RDR rules are made on it) - any traffic about it is handled by pf rules though
when a pot is created, all the relevant pf rules are written to a hook, so that

external traffic coming in over the compute interface is natted correctly into the pot
internal traffic (e.g., coming from consul) is routed over a transfer network between the reflect jail and the host (`route add "$APP_IP" <IP_ADDRESS>)
the reflect jail uses internal NAT to talk to the actual pots (see make-reflect-jail.sh)
the pot can reach a few services provided by the host over nat rules as well

Hosts participating in the cluster use static routes (e.g., route add <IP_ADDRESS>/32 <IP_ADDRESS>)

Now, these are a lot bits and pieces, which might cause many more questions - I hope the general concept is clear. If it is, we could see how we can streamline this into an example that works out of the box. Besides all the syntactic complication, the underlying concept is actually quite simple.
Traffic flow consul:
consul -> pot-reflect-jail -> nat -> rdr -> actual pot

Traffic flow internal:
src pot -> pot-reflect-jail -> nat -> rdr -> dest pot

Traffic flow external:
external -> compute_if -> rdr -> actual pot

It's been a while since I wrote it and it has been working stable in production for about two years now. Main advantages are:

You can scale services easily (x instances on y nodes)
You actually detect consul issues (like, if the DNS entry in consul does not match what is real, service checks will fail - a /etc/hosts based solution would mask the failure)

All of it is part of a bigger setup, which also features inbound and outbound load balancers and inter-service authentication using rotating client certificates - all of it is left out here for simplicity.
If you could share your basic setup, we could think about how to turn it into something like this without all the "big environment" elements, which make our setup potentially too complex for your needs.

@einsiedlerkrebs So yeah, if you could share the essence of your setup, I could see if our approach could be scaled down to meet your needs and then maybe even prepackaged, so it's useful to many.

Cool. I will write down something tomorrow.
On 30 April 2024 17:03:26 CEST, grembo @.***> wrote:

@einsiedlerkrebs So yeah, if you could share the essence of your setup, I could see if our approach could be scaled down to meet your needs and then maybe even prepackaged, so it's useful to many.

Hey @grembo,
thanks for the insights to your setup. It looks very interesting, sophisticated and comfortable. Happy to have tickle it out finally. Maybe it should get a more prominent location than this issue here.

@einsiedlerkrebs So yeah, if you could share the essence of your setup

The setup is very basic and straight forward.

Nomad as host service
consul in a manual pot (in order to meet nomad's dependency)
network/port based applications running in orchestered pots (via nomad jobs)
port service checking via consul based on hostnames

Since the highlevel nomad/consul stack is not aware of IP addresses chosen by the pot network when starting a pot it is no option to have the consul checks against an IP address that sits in the Nomads environment variables.
pots hostnames are suffixed by uuid's which are unknown as well

My setup is, that a service is reachable under the task name. The task name is set as an alias in the nomad-pot task configurations. The alias configuration is red by potnet and can as an option been included into the etc-hosts generation.
Pots can be flagged with dynamic-etc-hosts which sets their /etc/hosts file to be updated.
When Nomad/Pot starts or stops a services, it checks for all pots, if they are set to have dynamic hosts enabled and adds/removes the /etc/hosts/ entries accordingly.
When a pot-service came up, the /etc/hosts file in consuls pot is synced and it can check for the service by taskname which resolves to the IP inside the pot network.
No redirects, no extra (ncat) processes.
This approach is definitely something else, as what you described about your setup, but for a small single host setup it still brings a lot of pots/nomad/consuls comfort and to me it also looks as a clean and transparent way.
I would of course be happy, if you would see my approach useful enough (at least for me and others), to get it merged into the necessary components.

@einsiedlerkrebs It would be good if you could be more specific about your setup (like, give one or two example job configurations, how they interact etc.), as I would like to replicate your setup locally and see what we can do to make it work smoothly. The pure /etc/hosts workaround is nice, but IMHO in its current form it is too limited, as it will break a couple of nomad's features. I think your setup is a good real-life sized example we could derive from though, so maybe taking all the input from this thread + a good example setup, we could forge a solution that is more general purpose (or at least write some good howtos).
Just to confirm:
Your main concern is to be able to reach services from the jailhost (primarily, but not only, for consul checks), correct?

Okay. I will try.
Nomad is running on the host
consul
I am running consul manually:
set sysver=13.2.2 build=117 template=consul_template ipaddr=<IP_ADDRESS> name=consul
pot prepare -U file:///opt/pot-images/ -p $template -t ${sysver}.${build} -c "/usr/local/bin/cook-${name}" -N public-bridge -a ${build} -i ${ipaddr}  -n ${name} -v
pot mount-in -r -d /opt/pot-mounts/${name} -m /config -p ${name}_${build}
pot mount-in  -d /opt/pot-mounts/${name}data/ -m /data -p ${name}_${build}
pot set-attribute -A dynamic-etc-hosts -V YES -p ${name}_${build}
pot set-attribute -A no-etc-hosts -V NO -p ${name}_${build}
pot export-ports -p ${name}_${build} -e tcp:8500:8500 -e udp:8600:8600
pot info -vp ${name}_${build}
pot run ${name}_${build}
# ...

(the cook script runs consul)
some service (http)
job "dev" {
  datacenters = ["dc1"]
  type        = "service"
  reschedule {
    # this disables rescheduling
    attempts  = 0
    unlimited = true
  }
  group "dev" {
    count = 1

    restart {
      attempts = 3
      delay    = "15s"
      interval = "1m"
      mode     = "delay"
    }

    network {
      # https://developer.hashicorp.com/consul/docs/install/ports
      port "sensor_proxy" {
        static = 80
      }
    }

    task "sensorProxy" {
      driver = "pot"
      service {
        address = "<IP_ADDRESS>"
        meta {
          NOMAD_alloc_id = "${NOMAD_ALLOC_ID}"
          NOMAD_task_name = "${NOMAD_TASK_NAME}"
        }
        tags = ["sensors", "transport"]
        name = "sensorProxy"
        port = "sensor_proxy"
        check {
          type = "http"
          name = "http for sensors"
          port = "sensor_proxy"
          path = "/"
          interval = "30s"
          timeout = "1s"
        }
      }
      config {
        image        = "file:///opt/pot-images/"
        pot          = "access"
        tag          = "{{ sysver_major }}.{{ sysver_minor }}.{{ sysver_patch }}"
        command = "/usr/local/bin/cook-sensors"
        network_mode = "public-bridge"
        port_map = {"sensor_proxy" = "80"}
        mount_read_only = [ "/opt/pot-mounts/sensors:/config" ]
        attributes = [
          "localhost-tunnel:NO",
          "extra-hosts:YES",
          "dynamic-etc-hosts:YES",
          "no-etc-hosts:NO"
        ]
      }
    }
  }
}

Next are application containers. That are depending on the http service and can reach it by name (alias). This can already be tested within the consul pot.
pot run consul_117
(inside the consul)
ping sensorProxy.local

consul can check against a name as well:
HTTP GET http://sensorProxy.local:80/: 204 No Content Output:

Thank you for sharing.
Questions:

So do I understand correctly, that you're running consul in a pot?
What does your network config look like (/etc/rc.conf should probably be sufficient)?
For the sake of simplicity: I assume sensorProxy could be replaced by a plain nginx server, correct?

Hi @grembo ,
after the time passed, I am curious if there are any news.
Wish you a pleasant day and thanks.
einsiedlerkrebs

HI, just as an update, I spend a few hours last week to prepare a small example, but it's not self-contained/reproducible yet.

Hi. I am happy to hear! Do you have questions or can I help out with something?


bug: omit is missing from tailcallrc

.tailcallrc.graphql doesn't contain information about the omit operator. The operator is documented right here https://tailcall.run/docs/operators/omit/

/bounty 50$

/attempt 1339


feat(jwt) add support to a custom header name

Summary
It allow the jwt plugin to work together with another authentication
method that uses the 'Authorization' header
Reference: https://discuss.konghq.com/t/custom-header-on-jwt-plugin/1524
Full changelog

Add a header_names field to jwt plugin schema;
Change the retrieve_token method to  use the header_names field;
Add a test to check if the custom header_names is used;
Bump jwt plugin version from 0.2.0 to 0.2.1.

@Tieske I've fixed the minor changes and change the field to an array. Since I've rebased this branch on next, should I reopen this pull request to the next branch?

@raelmax at the top, right of the PR title is an edit button. It allows you to change the target branch. I already changed it to next.

Hmmm just noticed CI seems to be gone from this PR... Maybe better to indeed close this and open a new PR. (not sure that was the cause, but can't think of anything else)

Should I close this PR and open another one?

@raelmax Please keep the same PR.m, but rebase your patch on top of the next branch and force push your changes.

Hey guys, any update about the merge of this pull request? =)

@raelmax Hi,
At the moment we are busy with the 1.0 release, and we are delaying all unrelated work (including feature PRs). Sorry about the delay, once 1.0 out, we'll get back to this PR. Thanks.

Hi guys.
Are there any plans to move forward with this PR.
This feature is highly required.
Thank you in advance.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

replaced by #4757

@raelmax, Thank you for your contribution and apologies for the delay on our end. It has been merged (thanks @Tieske) and will be in Kong 1.3.


Setting metadata for .exe file

Is it possible to include a feature that allows users to set metadata for the compiled file, eliminating the need for manual editing with Resource Hacker or similar tools?

Yes it's planned 😉


Move to lightweight Docker image

The Dockerfile has been changed to produce a smaller image (22mb vs 764mb) that runs as user exporter.
If this is accepted, please consider adding the image to Docker hub.

Thank you for the PR!

I've created a Docker Image mesosphere/mesos_exporter and set up automated builds at https://hub.docker.com/r/mesosphere/mesos_exporter/


Fix extra letters bypass problem

What's the problem this feature will solve?
The fact that some profanity is not caught because some letters or numbers are added to the end of it. For example ghtprofanityght the profanity is there but not caught by the filter because of extra letters.
Describe the solution you'd like
Using REGULAR EXPRESSIONS special characters to find any profane words even with any letters or numbers added to it.

As far as I can tell, the code of warehouse itself doesn't define profanity words, we just have a list of prohibited project names in the database, which happens to include offensive words. While we could make our search a bit more exhaustive, it would make a lot of sense in many cases.
For example, we may want to ban the name request because it's too close to requests, but we don't want to base requests itself, or a new package named requests-async or requests-http3 or something (a third party package). If we wanted to do this, we would need to actually add in the DB a column scpecifying which names are forbidden because they're offensive and which ones because they look too much like existing package (and there's also copywrighted stuff etc), and then make special rules for querying this table. Also, we currently have ~26000 entries in the table, which we would need to categorize somehow. We probably wouldn't have to do it 100% by hand, but I believe this would be quite some work.
And even if we would do all that, people would still be registering prof-anity and pr0f4ni7y and prof*ckingfanity and proofanity and profinaty and more subtle racist dogwhistle slur with a plausible deniablity and ...
It's a never ending game.


feat(releasing): New task to automatically create release schedule

What does this PR do?
This new task create the release page and the associated notes.
A potential improvement would be to create the google calendar entries
as well.
For now to be launched locally with atlassian credentials, could be added as a workflow on github

Motivation
Automation

Additional Notes
cc @kacper-murzyn

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


Feature: notification on player join

Classic scenario:

I create a game
I'm alone
nobody joins
I start lurking the Internet and scroll among thousands of pictures of pugs
while I fall in love with cuty-pie, a 2 yrs old cute pug, someone joins my room and set ready
after few minutes he leaves the room
I come back and I'm still alone and sad in the room

It would be awesome to have sound + tab notification to warn players when someone joins.

Done!


clarifying that these are web frameworks

#sign-off

Docs Build status updates of commit aa691d0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/static-web-apps/overview.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

#sign-off


Choose a sort order of directories and files according to the settings pattern

VSCode Version:
Version 1.10.2
Commit 8076a19fdcab7e1fc1707952d652f0bb6c6db331
Date 2017-03-08T14:00:46.854Z
Shell 1.4.6
Renderer 53.0.2785.143
Node 6.5.0

OS Version:
Distributor ID: LinuxMint
Release: 18.1
Codename: serena

Choose a sort order of directories and files according to the settings pattern. For example:
Based on this ASCII table from the 32 character

// ASCII pattern
"explorer.pathsSortOrders": [
    " ",
    "!",
    "#",
    // ...
    "@",
    "A",
    "B",
    "C",
    // ...
    "a",
    "b",
    "c",
    // ...
    "{",
    "|",
    "}",
    "~"
],

P.S.: Some IDE like phpStorm follows the above ASCII standard to sort directories and files of a project.

Will this be implemented?

I was about to file the same issue. Ideally, I'd like my test folder next to the files they test, unlike this example:

Wtf? Why duplicated?


ModuleNotFoundError: No module named 'setup_run_dir'

Hi,
Happy New Year,
I want to try to train a model, After running 20_Assemble_LDC2020T02/10_CollateData.py and 20_Assemble_LDC2020T02/12_TestLoadPenman.py, I can get

And then I want to run 30_Model_Parse_GSII/10_Annotate_Corpus.py but got a ModuleNotFoundError: No module named 'setup_run_dir' error

This error repeat like the image above, but actually I can find setup_run_dir.py in 30_Model_Parse_GSII.
When I comment these code:

for fn in ('test.txt', 'dev.txt', 'train.txt'):
annotate_file(indir, fn, outdir, fn + '.features')

The error disappears but nothing happened, Did I make a mistake in one of my steps, or do you have any possible solutions?
Thanks a lot in advance!

I don't know why you're getting that error.  The best thing to do would be to move the script your running (ie 10_xx) up 2 directories and comment out the line at the top with import setup_run_dir.py.  As the comment on that line states, all the import of that module does is to force python into running the script 2 levels up. This is just so I can keep things organized in separate directories.  If you manually move the file up 2 directories the paths for amrlib and the data should all be in the right locations without having to do the import.

Thank you very much for your reply,
I tried to move the script up 2 directories and comment out the line at the top with import setup_run_dir.py.
It still fails, with showing the following multiprocessing.pool.RemoteTraceback error, but actually I just run the script on local.

How can I deal with this?

Are you using Windows?  Multiprocessing and Windows don't play well together so that might be the issue.  Try this...
Go to annotator.py and un-comment line 26 then comment out line 25, 27, 29 and 30. This will remove all the multipocessing.Pool() stuff and replace it with a single-threaded map call.
Let me know if that works and if so, I'll try to figure out how to if/else these statements so if anyone else is using Windows they don't run into this again.

No , I am using Mac. I will try that now.
btw, load_spacy('en') the en model here , does it mean en_core_web_sm model from spaCy or it is a model which should be downloaded following the instruction in SpotlightDBServer.sh ? Do I need to set up the JAVA environment In addition to running requirements.txt ？

Oh , it works

On the multiprocessing failure...
Interesting.  Per the docs... "Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess."
Note to self--> use multiprocessing.get_start_method() == 'fork' to check for this and disable pool code if not equal . There is also os.name and sys.platform if the above causes any issues.

On Spacy loading...
The en model used to be the default model and could change depending on how you setup your system (ie.. it was a pointer to the actual model).  You might want to just change this to en_core_web_sm so you know which model it's using.
The SpotlightDBServer is a whole different thing.  It's only needed if you want to add the wiki tags to the graphs (done in 32_Add_Wiki.py).  This is a separate post-processing operation that most people skip.  If you really need this, you'll have to follow the instructions to set it up, including any of the JAVA stuff.  Be aware, this is an old system and it may be a pain to get working.
If you really need the wiki tags, I suggest using the newer BLINK system.  The script for that can be found in 33_Model_Parse_XFM/30_Add_Wiki_With_Blink.py.  It still takes some work to setup but it gives better results.


Jackett: Wrong port for the WebUI

Hi, thanks for all your addons !
I installed Jackett a few days ago, everything was okay, but a recent update changed the web ui port from 9117 to 8889, as we can see in the logs below :
Now listening on: http://[::]:8889
Application started. Press Ctrl+C to shut down.

But inside the config.json file, it is still set to 9117.
I forked the repo and changed port from 9117 to 8889 in config.json and it is now working.
Maybe the best solution would be to make Jackett run on 9117 as it was before instead of changing the exposed port.
Thanks in advance.
I am running the latest version of Home Assistant OS on a Raspberry pi 4.
(Also, in the Readme file, there is a little mistake about the webui port : Webui can be found at <your-ip>:7878.)

Hi,
I'm sorry but the port is still 9117. I've installed a clean instance and here is the log :
Hosting environment: Production
Content root path: /app/Jackett/Content
Now listening on: http://[::]:9117
Application started. Press Ctrl+C to shut down.

Most probably you have changed the port from the Jackett ui...
The easiest would be to restore a backup if you have!
Thanks for the readme, I'll update that

Hi,
I checked my config.json file in /config/Jackett and the port was still 9117.
But there was another config.json file in /config/addons_config/Jackett and here the port was not 9117, I deleted that one and it came back to 9117 after addon restart, so you were right I obviously messed up with the port but I really can't remember that.
Sorry for bothering you

No problem, glad it worked!


Broken conda upload

Looking at the CI build for the latest commit (https://github.com/coiled/coiled-runtime/commit/3d3761890cfe3ce1e7cfcf38f1e2dc9dc35d779e) we're failing to upload the coiled-runtime metapackage due to the auth token we're using lacking appropriate permissions
...
Using Anaconda API: https://api.anaconda.org/
Error:  ('Authentication token does not have the sufficient scope to perform this action expected: api:read', 401)
Error: Process completed with exit code 1.

cc @hayesgb who I believe made recent changes related to this

Linking to a related comment from Greg here https://github.com/coiled/coiled-runtime/commit/3d3761890cfe3ce1e7cfcf38f1e2dc9dc35d779e#commitcomment-70739508

The COILED_CHANNEL_TOKEN failed with insufficient privIleges. This was corrected by updating to COILED_UPLOAD_TOKEN

Closing as we just re-issues the coiled-runtime=0.0.3 release to the coiled conda channel using our CI infrastructure https://github.com/coiled/coiled-runtime/commit/eee995ce4e22d939e56b61a198e496a629ca429d


Realtime feedback?

Is it possible at the moment (or as a future feature) to get realtime feedback whilst Vegeta is running?  I'm running tests over multiple hours and being able to get a rough idea of the count of status codes returned or average times would be tremendous.
NB: a genuinely brilliant project - helping me no end.

Hi @tsenart
I know you are reticent of adding new features, but I also see realtime feedback as important. I was thinking of response time percentiles, current RPS and error rate printed to stdout every x number of seconds. If we could also format these metrics as Graphite data, we could push this data to InfluxDB and visualise them through Chronograf.

I'll experiment with piping the results binaries into another instance of vegeta later today and shout back if that's doable or not.

This works fine.  In another session cat the output and pipe into "vegeta report" and you get running stats.  Shouldn't be that hard to ask for them as JSON and then pipe into Statsd/whatever

@sage-oli-wood: Glad you found your way around. @aidylewis: You can use vegeta dump for your use case I think.

yes, that's a good idea by @sage-oli-wood.
shell_1
echo "GET https://www.google.co.uk/" | vegeta attack -duration=2m -rate=2 | tee results.bin | vegeta report

shell_2
while true; do cat results.bin | vegeta report ; sleep 2; done

Thanks

And of course the dumper works as well. This tool is extremely well designed.


Unable to work and how to set border radius in this toast?

Fluttertoast.showToast(
msg: "This is Center Short Toast",
toastLength: Toast.LENGTH_SHORT,
gravity: ToastGravity.CENTER,
timeInSecForIosWeb: 1,
backgroundColor: Colors.red,
textColor: Colors.white,
fontSize: 16.0
);

Platform support is limited not able to add border radius


KPointsData rich view

I adapted the npm package https://github.com/materialscloud-org/brillouinzone-visualizer to conditionally render the various components based on what data is available (see v0.4.0). the only mandatory data are the reciprocal lattice vectors (b1, b2, b3).
The implementation should

access the  /contents/derived_properties endpoint (see e.g. https://aiida.materialscloud.org/mc3d/api/v4/nodes/a9e1bcdc-2843-4876-bb82-c8fce4485c2f/contents/derived_properties);
check if cell is available has_cell and saves faces_data;
gets cell information from /contents/attributes (see e.g. https://aiida.materialscloud.org/mc3d/api/v4/nodes/a9e1bcdc-2843-4876-bb82-c8fce4485c2f/contents/attributes)
calculates (b1, b2, b3) based on the cell information
passes (b1, b2, b3) and faces_data to the BZVisualizer.

The old implementation does something similar: https://github.com/materialscloud-org/aiida-explorer/blob/85afc172369b05e085ff825af96f0bc74e661ce2/app/scripts/controllers/explore/nodedetails/data/kpoint.js#L74-L124
You can just copy where there on how to calculate (b1, b2, b3).
Old example, for comparison
@SharanRP
Note: might be better to prioritize other things if this seems complicated.

use the latest version of brillouinzone-visualizer


Fix option values that include =

Why
Options that include = inside their value would break, in particular langmaps:
set langmap=ab,cd,=e should remap = to e, instead the option value was cut early at ab,cd,.
What changed
Merge all parts of the split string string but the first one using =.
Test plan

:set langmap==j
=
Evaluate whether the cursor has moved down by one line

Thank you!


Add minimal Typescript example with repeatable fields

I'm having a really tough time getting typescript to be ok with my slices.  The custom-slicezone-props example was helpful, however looping through nested items I can't seem to figure out.

Hey @beamercola, could you provide a short example of what you're trying to do? I can write up an example if you point out what isn't working.
(By the way, I'm working on a CLI that will generate TypeScript types for you automatically that you might be interested in. We'll share publicly once it's available - hopefully soon.)

If you're looking for an example of how to type the repeatable fields in a Slice, you can provide the fields as the second type parameter to prismicT.Slice:
export type HeroSlice = prismicT.Slice<
	"hero",
	{
		heading: prismicT.KeyTextField;
		text: prismicT.RichText;
	},
	{
		buttonLink: prismicT.LinkField;
		buttonText: prismicT.KeyTextField;
	}
>;

This will type the items field as an array of objects with those properties.
(In addition to the CLI I'm working on, we're also working on writing/generating documentation for the TypeScript Types. They aren't documented today and recognize that documenting them will be helpful.)

@beamercola I added a TypeScript-specific example here: https://github.com/prismicio/prismic-react/tree/master/examples/with-typescript
I'm going to close this issue, but please let me know if you are looking for something specific in the example. I'm more than happy to update it. :)

Amazing, thank you!


feat: Add string support to the transform property

Summary
This updates the transform property to support string values as requested on https://github.com/facebook/react-native/issues/34425. This also updates the existing unit tests of the processTransform function ensuring the style processing works as expected and updates the TransformExample on RNTester in order to facilitate the manual QA of this.
Changelog
[General] [Added] -  Add string support to the transform property
Test Plan

Open the RNTester app and navigate to the Transforms page
Check the transform style through the Transform using a string section

https://user-images.githubusercontent.com/11707729/189550548-ee3c14dd-11c6-4fd1-bd74-f6b52ecb9eae.mov

@gabrieldonadel / @necolas I just tested in 0.71 branch and in RNTester, when I try to the test page, it crashes:

I hit this when integrating 0.71 into react-native-windows.  Our automation hit this crash, so I've had to manually disable this test page in our tests for now.   The main thing I want to be sure of is that any transform value that was valid before this change is still valid.  -- Maybe we want to enforce units on transforms going forward, but failures there should be yellowboxes not invariants during the transition period.  Otherwise, it will be very hard for any package that wants to support multiple versions of RN.

The main thing I want to be sure of is that any transform value that was valid before this change is still valid.

That shouldn't be a problem as we're only requiring units for string values, everything that was valid before is still going to be valid.

Thanks for the report @kelset, I've just opened a PR fixing this https://github.com/facebook/react-native/pull/35292.
This also fixes your problem @acoates-ms

failures there should be yellowboxes not invariants during the transition period

Yes I think invalid values should be yellowboxes, mostly because soft failures with warnings are better for styling than crashes. But we still shouldn't allow invalid syntax/values to work as if it were valid


Management State Question

So I just had to rebuild my cluster after the rancher/rancher container kept crashing.  Sadly I don't have any of the error logs but after troubleshooting I'm have reason to suspect that the management state became corrupt/bloated as the server running rancher was under considerable load.  So my question is the management state able to be scanned/rebuilt over time?
Some context, we need to regularly rotate out our instances to address OS vulnerabilities so the management state likely has old instances in it that no longer exist.

Any updates?


Add files via upload

Docs Build status updates of commit 9099031:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

windows-iotcore/commercialize-your-device/MS EULA_Windows 10 IoT Core RS5 .docx
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit f6fb64f:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

windows-iotcore/commercialize-your-device/MS EULA_Windows 10 IoT Core RS5 .docx
:white_check_mark:Succeeded

windows-iotcore/commercialize-your-device/SAC-Agreement-RS5.pdf
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


socket io work only in localhost

i already try your code, it was great at firts, it can show chart data from arduino properly on localhost:4000 but when i try to access it (localhost) with <IP_ADDRESS>:4000 (which is the ip of my localhost laptop) from another computer i can still show the front end interface, but the moving chart didnt show and from chrome console it show error like:
polling-xhr.js:264 GET http://localhost:4000/socket.io/?EIO=3&transport=polling&t=MO6fy14 net::ERR_CONNECTION_REFUSED

This is happening because the socket.io server is running on <IP_ADDRESS>:4000 but socket.io-client is trying to connect to localhost:4000. To make it work, you must replace localhost:4000 with <IP_ADDRESS>:4000 in index.html file.
var socket = io.connect('http://<IP_ADDRESS>:4000');


Create AXI top-level for BP; starting with FPGA use case

Main deliverables are
[ ] CCE->AXI-4 adapter
[ ] CCE->AXI-lite adapter
[ ] Top-level, "AXI-only" wrapper for use in FPGA
[ ] System-level diagram such as this

Add: AXI-4 to CCE
Add: AXI-Lite to CCE

#684


Document how to enable read only mode

I wanted to run this in read only mode and it took me a while to figure out I just need to add the following to config/nginx/location.conf, this might be useful for other users.
      limit_except GET PROPFIND OPTIONS HEAD {
            deny all;
      }

Excellent, thank you for sharing! I will close this.

Just a idea could you add this to the readme section?
TIA

Just a idea could you add this to the readme section? TIA

I am conflicted about this for two reasons:

It opens a can of worms. Different users have different needs. It is not the aim of this project to instruct users on nginx configurations. limit_except is part of ngx_http_core_module. There are tons of other stuff in there.
I do not want to potentially provide insecure configurations as part of the documentation.

What I can do, however, is pointing users to nginx documentation.


Merge Editor: show a diff in base like as if I would compare one side to base

once base is visible, can we not show a diff like as if I would compare one side to base? Maybe take the active focussed side and update diff markers?
here is what I see today in diff and that helps me to understand the reasons for a change:

Originally posted by @bpasero in https://github.com/microsoft/vscode/issues/160247#issuecomment-1239159022

👍 , I think the interesting bit is that the diff would show for the active side (Incoming, Current) because I think we cannot really show a meaningful diff of 3 editors compared to each other?

Oh and it would probably require Base to be in between Incoming and Current

I think we cannot really show a meaningful diff of 3 editors compared to each other?

Not really I think.
I like this idea!

Closed in https://github.com/microsoft/vscode/pull/163553

Does the diff only show the part most relevant to the merge conflict?
In the screenshot below, base has a function parseBindingElement, which seems to be renamed to parseArrayBindingElement in Input 2, but the diff isn't highlighted for those lines.

The diff is only shown for proper conflicts. You can show all conflicts through the ... menu.


Aligned bottoms of buttons in back up sequence

Closes #4090

Before

After

@eviltofu much better. Thanks.
The bottoms of the 1 button on the left and Clear button on the right aren't vertically aligned. Is that intentional?

@eviltofu much better. Thanks.
The bottoms of the 1 button on the left and Clear button on the right aren't vertically aligned. Is that intentional?

The button bar is aligned properly but the buttons inside the UIStackView always puts a separating space below the buttons. I can't figure out which combination of fill and layoutMargins are needed to flush it against the bottom. The other alternative is to make my own stack view. @hboon

@eviltofu much better. Thanks.
The bottoms of the 1 button on the left and Clear button on the right aren't vertically aligned. Is that intentional?

The button bar is aligned properly but the buttons inside the UIStackView always puts a separating space below the buttons. I can't figure out which combination of fill and layoutMargins are needed to flush it against the bottom. The other alternative is to make my own stack view. @hboon

I see. Never mind. It doesn't look obvious unless aligned side by side.


Unable to Receive Push Notifications when App is closed

Hi there,
I was finally able to get my parse-server and Unity SDK working. The last piece of the puzzle is fixing push notifications for Android.
So far, It works when the app is opened or in the background but throws an error when the app is is in the background.

07-29 15:52:24.781  3522  3537 W Unity   : Native libraries not loaded - dropping message for ParseInitializeBehaviour.OnPushNotificationReceived

My Manifest file is:
    <service android:name="com.parse.ParsePushService" />
        <receiver android:name="com.parse.ParsePushBroadcastReceiver" android:permission="com.google.android.c2dm.permission.SEND">
            <intent-filter>
                <action android:name="com.parse.push.intent.RECEIVE" />
                <action android:name="com.parse.push.intent.DELETE" />
                <action android:name="com.parse.push.intent.OPEN" />
                <action android:name="com.google.android.c2dm.intent.RECEIVE" />
                <action android:name="com.google.android.c2dm.intent.REGISTRATION" />
                <action android:name="android.intent.action.BOOT_COMPLETED" />
                <action android:name="android.intent.action.RECEIVE_BOOT_COMPLETED" />
                <action android:name="android.intent.action.USER_PRESENT" />
                <category android:name="com.xxxx.xxxxx" />
            </intent-filter>
        </receiver>

<meta-data android:name="com.parse.push.gcm_sender_id" android:value="id:882389752921"

/>

I think this is an issue with how the Parse Plugin handles Notifications from GCM but I cant seem to place my finger on the issue. Can someone please have a look?

I believe you need to add a GCM broad cast receiver.

Just forget about using .net parse with parse server push is not working if its closed. the broad cast receiver methods are not working.
Till this is fixed its not a good idea to use. Reversing everything back to normal parse hope that in the coming half year the sdk is updated to be used properly with parse server

I was getting the same issue and managed to fix it by changing the code of the ParsePush.jar a bit. What you got to do is simply change the initial value of applicationPaused in ParsePushUnityHelper.java from false to true.
Here is the compiled jar: ParsePush.zip
Tested with Unity 5.5.2 and a Parse.Unity.dll downloaded from #166

Push is working fine, it has also change a lot since the server was first made available as OSS. Marking as this needs followup and may be a potential bug in this sdk, independent of the server.


Adopt phantomjs-prebuilt

Phantomjs now has a node wrapper that installs platform specific libraries if they are not present.
It would be awesome to make phantomjs-node use those libraries as it would completely remove the need of manually installing phantomjs.

+1 for this. It will be great if there is not need to have phantomjs installed already. phantmjs-prebuild is awesome, it should be easy to just point this module to that binary(i.e. include it as a dependent module)

Agreed. If you look at package.json there is   "phantomjs": "~1.9.7-5" in devDependencies. So it's already there. It's just a matter of moving to just dependencies. @cdaniel want to issue a pull request?

Looks like the bin path needs to be changed to phantomjs.path. If you make that change then that would be great. Otherwise, I have been doing a lot of work on this package. I should be able to get it in later this week.

I've created a pull request, although apparently not in this bug https://github.com/amir20/phantomjs-node/pull/357


add 'nrel' spa option for ET irradiance, fix doy - 1 bug

This PR adds a new 'nrel' option to irradiance.extraradiation that uses the well tested ephemeris code in spa.py. It also fixes the doy - 1 bug described in #211.
My plan is to let #214 sit for another day for comments, merge it, and then rebase/merge this PR.

Here are some benchmarks:
times = pd.DatetimeIndex(start='2015', end='2016', freq='1min')

In [40]: %timeit pvlib.irradiance.extraradiation(times, method='nrel', how='numba')
1 loop, best of 3: 1.67 s per loop

In [41]: %timeit pvlib.irradiance.extraradiation(times, method='nrel')
1 loop, best of 3: 3.87 s per loop

In [42]: %timeit pvlib.irradiance.extraradiation(times, method='pyephem')
1 loop, best of 3: 8.34 s per loop

In [43]: %timeit pvlib.irradiance.extraradiation(times, method='spencer')
10 loops, best of 3: 45.8 ms per loop

In [44]: %timeit pvlib.irradiance.extraradiation(times, method='asce')
10 loops, best of 3: 26 ms per loop

I was hoping that the nrel method would be faster, but it's still an improvement over pyephem.

A sloppy copy/paste job led to an unnecessary slow line of code in solarposition.nrel_earthsun_distance. I removed the slow line and reran the tests:
In [61]: times = pd.DatetimeIndex(start='2015', end='2016', freq='1min')

In [62]: %timeit pvlib.irradiance.extraradiation(times, method='nrel', how='numba')
The slowest run took 10.33 times longer than the fastest. This could mean that an intermediate result is being cached.
1 loop, best of 3: 215 ms per loop

In [63]: %timeit pvlib.irradiance.extraradiation(times, method='nrel')
1 loop, best of 3: 953 ms per loop

In [64]: %timeit pvlib.irradiance.extraradiation(times, method='pyephem')
1 loop, best of 3: 7.9 s per loop

In [65]: %timeit pvlib.irradiance.extraradiation(times, method='spencer')
10 loops, best of 3: 64.4 ms per loop

In [66]: %timeit pvlib.irradiance.extraradiation(times, method='asce')
10 loops, best of 3: 28.5 ms per loop


Update cwltool for 1.7.0

┆Issue is synchronized with this Jira Story
┆Issue Type: Story
┆Fix Versions: Dockstore 1.7
┆Sprint: Seabright Sprint 16 Pram
┆Issue Number: DOCK-817
┆Epic: Update cwltool/cromwell 1.7.0

https://github.com/dockstore/dockstore/pull/2690


Many missed eth1deposit blocks in t_metadata

I'm running chaind with the following start script, which reading data from a local ethereum node (geth + teku)
chaind  --eth1deposits.enable true \
        --eth1client.address "<geth rpc>" \
        --eth2client.address "<teku api>" \
        --chaindb.url "<postgresql port>" \
        --log-level "info" \
        --log-file "./chaind.log" \
        --beacon-committees.enable true \
        --blocks.enable true \
        --finalizer.enable true \
        --proposer-duties.enable true \
        --summarizer.blocks.enable true \
        --summarizer.enable true \
        --summarizer.epochs.enable true \
        --summarizer.validators.enable true \
        --validators.balances.enable true \
        --validators.enable true

Both clients have finished sync, however chaind still records a lot of missed blocks in t_metadata table, see the following
schema,{"version": 8}
blocks.standard,{"latest_slot": 63588}
proposerduties.standard,{"latest_epoch": 24195}
eth1deposit.getlogs,{"latest_block": 11554048\, "missed_blocks": [11185281\, 11185282\, 11185283\, 11185284\, 11185285\, 11185286\, 11185287\, 11185288\, 11185289\, 11185290\, 11185291\, 11185292\, 11185293\, 11185294\, 11185295\ ...... it's too long
synccommittees.standard,{"latest_period": 486}
beaconcommittees.standard,{"latest_epoch": 21825}
validators.standard,{"latest_epoch": 160917\, "latest_balances_epoch": 5404}

t_eth1_deposits table also record nothing, I can read blocks in the missed_blocks list from geth

Are there any warnings or errors in the chaind log regarding the failure to read the deposits from the execution node?

Yes, here's part of the error and warning message:
{"level":"warn","service":"eth1deposits","impl":"getlogs","start_block":12987969,"end_block":12988032,"error":"no transaction returned for hash 0x738c8873aec054ac980e7bc188750332727887234081f1c158719c23f09bc36c","time":"2022-11-18T13:01:38+08:00","message":"Failed to update ETH1 deposits"}
{"level":"warn","service":"eth1deposits","impl":"getlogs","start_block":12988097,"end_block":12988160,"error":"no transaction returned for hash 0x3c21811401eb90d863c0c15965628834ef98be1f7e6286d0efce9180e6fd5158","time":"2022-11-18T13:01:42+08:00","message":"Failed to update ETH1 deposits"}
{"level":"warn","service":"eth1deposits","impl":"getlogs","start_block":12988225,"end_block":12988288,"error":"no transaction returned for hash 0xf6cb8c6191e2ba048b89d84b0ec5f8de2e60a9607df9fc73df2d65879a83b7b8","time":"2022-11-18T13:01:45+08:00","message":"Failed to update ETH1 deposits"}
{"level":"warn","service":"eth1deposits","impl":"getlogs","start_block":12988225,"end_block":12988288,"error":"no transaction returned for hash 0xf6cb8c6191e2ba048b89d84b0ec5f8de2e60a9607df9fc73df2d65879a83b7b8","time":"2022-11-18T13:01:47+08:00","message":"Failed to update ETH1 deposits"}
{"level":"warn","service":"eth1deposits","impl":"getlogs","start_block":12988289,"end_block":12988352,"error":"no transaction returned for hash 0xeea6786dae8596b10f1f3a741ba7d396b1b0b68ca01761e6afc562a20dece0c4","time":"2022-11-18T13:01:47+08:00","message":"Failed to update ETH1 deposits"}
{"level":"warn","service":"blocks","impl":"standard","slot":124117,"error":"failed to set block: ERROR: duplicate key value violates unique constraint \"i_blocks_1\" (SQLSTATE 23505)","time":"2022-11-18T13:01:53+08:00","message":"Failed to update block"}
{"level":"warn","service":"eth1deposits","impl":"getlogs","start_block":12988289,"end_block":12988352,"error":"no transaction returned for hash 0xeea6786dae8596b10f1f3a741ba7d396b1b0b68ca01761e6afc562a20dece0c4","time":"2022-11-18T13:01:55+08:00","message":"Failed to update ETH1 deposits"}

I tried to read transaction from geth, eth.getTransaction method returns nothing but eth.getTransactionFromBlock works:
> eth.getTransaction("0x738c8873aec054ac980e7bc188750332727887234081f1c158719c23f09bc36c")
null

It seems I need to add --txlookuplimit 0 to allow looking up the entire chain transactions.
refer to this issus: https://github.com/ethereum/go-ethereum/issues/23569
now only block duplicate error left. I will rerun chaind from begining and close this issue if there's no more error.

Hi~ thanks
I'm not sure, do I need to add --eth1deposit.getlogs to the end of my start script?
the block error is because chaind is trying to write duplicated blocks, it has been solved by restarting chaind

New issues:
{"level":"warn","service":"synccommittees","impl":"standard","period":633,"error":"failed to fetch sync committee: failed to request sync committee: failed to call GET endpoint: Get \"http://<IP_ADDRESS>:5051/eth/v1/beacon/states/5185536/sync_committees\": context deadline exceeded","time":"2022-11-22T02:05:30+08:00","message":"Failed to update sync committee"}
{"level":"warn","service":"beaconcommittees","impl":"standard","epoch":93738,"error":"failed to fetch beacon committees: failed to request beacon committees: failed to call GET endpoint: Get \"http://<IP_ADDRESS>:5051/eth/v1/beacon/states/2999616/committees\": context deadline exceeded","time":"2022-11-26T13:12:46+08:00","message":"Failed to update beacon committees"}

I got new issues on fetching committees and sync committees from teku client. I try to curl committees, which need a long time to get the response, about 120 seconds. How can we solve this issue?
curl http://<IP_ADDRESS>:5051/eth/v1/beacon/states/3261760/committees

The teku documentation regarding archive frequency is relevant here: https://docs.teku.consensys.net/en/latest/Reference/CLI/CLI-Syntax/#data-storage-archive-frequency
You can increase chaind's timeout with the ethclient.timeout option.

Thank you very much, I solved my problem by setting data-storage-archive-frequency: 32 and --eth2client.timeout "3m0s"


add noto to firefox

Adds Noto fonts to the Firefox Dockerfile to fix the tofu I get while browsing.

thanks!


Retain github and linkedin social icons in mobile

Bug Title
Social icons in mobile view
Description of BUG
We need to have only GitHub and LinkedIn social media icons on all pages, as it doesnt look nice to have 6 icons for mobile view
This is only gor mobile view and the styling changes should be made in socialButton component tailwindCSS
Reporduction URL of bug
Nill
Bug Issue Type
Frontend
Screenshot of the issue if in UI/ Document

Are you taking this issue fix
Nill

Hello @jsvigneshkanna, I'm new here
can I work on this task while I try to familiarise myself with the code base?

Welcome @eddybruv
Go through the codebase and play around it, check how tailwind components are showcased
Let me know in any issues if you can take those too

@eddybruv
Please let me know if any help needed, this issue needs to fixed with priority as this reduces the UX of product

Just facing some power issues, should be resolved before evening, sorry 😅

No worries!


Add support for nanorc

Preliminary Steps
Please confirm you have...

[X] reviewed How Linguist Works,
[X] reviewed the Troubleshooting docs,
[X] considered implementing an override,
[X] verified an issue has not already been logged for your issue (linguist issues).

Problem Description

nano is a very commonly available console-based text editor for GNU/Linux and other systems. It's often installed by default, or easily installable using apt-get or similar mechanisms. Its configuration file is called ~/.nanorc.
It would be great if linguist would provide support (syntax highlighting) for nanorc files. This issue exists to track adding this support.
Here is a reasonably complete syntax definition that could be used if anyone feels like looking into this: https://github.com/perlun/dotfiles/blob/master/nano/syntax/nanorc.nanorc (this file in itself uses the nanorc syntax, so it can also be used as an example)
URL of the affected repository:
Example file: https://github.com/perlun/dotfiles/blob/master/nanorc
Last modified on:
2018-11-22
Expected language:
nanorc
Detected language:
None

This would actually be a good idea. Dotfile repos are extremely common on GitHub, and many hackers tend to dig through them looking for tricks. Syntax highlighting for .nanorc files would certainly reach many users.
By the same token, other .*rc files would also benefit from syntax highlighting, such as .curlrc. The only part that bothers me is that the addition would be purely aesthetic, and we don't even have highlighting/classification for .gitignore files (the single most common manifest file in Git repositories).
@pchaigno Any thoughts?

I'm all for it, but do we have a Sublime Text, Atom, or TextMate grammar we could use to highlight these files?

@pchaigno Maybe this could be of help?

https://github.com/otgo/highlight-nanorc (2 years old, but nano probably hasn't changed that much since)

Looked for TextMate or Sublime but couldn't find any.

I'm tempted to hack one together, because we could leverage our existing grammar for highlighting regular expressions — all we'd need is a wrapper for it and some .nanorc-specific keyword patterns. 👍
The otgo/highlight-nanorc grammar appears somewhat flaky. Notice the inconsistent highlighting.

I'm tempted to hack one together

@Alhadis How much time would you need? Should we use otgo/highlight-nanorc in the meantime?

Already finished. Lol...
Haven't published it yet. I'll do so now.

@Alhadis If you want to submit the pull request to use your grammar, go ahead, I don't think @perlun is going to...

Will do. Should I add the grammars I wrote for .gitignore, .gitattribute and .gitconfig files too? They're, uh, damn near airtight in their syntax and tailored specifically to GitHub's colour scheme...

:+1:
We should probably group the corresponding languages under a common parent language.

I guess INI will do. =)

@Alhadis If you want to submit the pull request to use your grammar, go ahead, I don't think @perlun is going to...

Yeah, sorry, I didn't see this until now and I'm pretty certain @Alhadis is a more experienced contributor to the project anyway. =) So, I'm happy to delegate the work in this case. 😉
Many thanks for looking into it! 👍


Added Croatian locale

I've created i18n json file for Croatian language, locale sign is "hr"

Thank you for your contribution @mZabcic!


Simulation crashed. I can't found the reason. Maybe Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

google colab run main
INFO :      Starting Flower simulation, config: num_rounds=100, no round_timeout
Initializing FedTabDiff model
/usr/lib/python3.10/subprocess.py:1796: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock.
  self.pid = _posixsubprocess.fork_exec(
2024-07-15 09:26:42,335	INFO worker.py:1752 -- Started a local Ray instance.
INFO :      Flower VCE: Ray initialized with resources: {'memory':<PHONE_NUMBER>.0, 'node:<IP_ADDRESS>': 1.0, 'GPU': 1.0, 'node:__internal_head__': 1.0, 'object_store_memory':<PHONE_NUMBER>.0, 'CPU': 2.0, 'accelerator_type:T4': 1.0}
INFO :      Optimize your simulation with Flower VCE: https://flower.ai/docs/framework/how-to-run-simulations.html
INFO :      No `client_resources` specified. Using minimal resources for clients.
INFO :      Flower VCE: Resources for each Virtual Client: {'num_cpus': 1, 'num_gpus': 0.0}
INFO :      Flower VCE: Creating VirtualClientEngineActorPool with 2 actors
INFO :      [INIT]
INFO :      Using initial global parameters provided by strategy
INFO :      Evaluating initial global parameters
INFO :
INFO :      [ROUND 1]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 2]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 3]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 4]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 5]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 6]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 7]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 8]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 9]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures
INFO :
INFO :      [ROUND 10]
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
Initializing FedTabDiff model
[Server evaluation, server round: 10
Loading eval set
SAMPLING STEP:  499: : 0it [00:00, ?it/s]
ERROR :     Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
ERROR :     Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/flwr/simulation/app.py", line 323, in start_simulation
    hist = run_fl(
  File "/usr/local/lib/python3.10/dist-packages/flwr/server/server.py", line 490, in run_fl
    hist, elapsed_time = server.fit(
  File "/usr/local/lib/python3.10/dist-packages/flwr/server/server.py", line 126, in fit
    res_cen = self.strategy.evaluate(current_round, parameters=self.parameters)
  File "/usr/local/lib/python3.10/dist-packages/flwr/server/strategy/fedavg.py", line 167, in evaluate
    eval_res = self.evaluate_fn(server_round, parameters_ndarrays, {})
  File "/content/cloned-repo/FlowerServer.py", line 49, in evaluate_server
    generated_samples = generate_samples(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/content/cloned-repo/fedtabdiff_modules.py", line 171, in generate_samples
    model_out = synthesizer(z_norm.float(), t, label)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/content/cloned-repo/MLPSynthesizer.py", line 148, in forward
    emb = self.time_embed(timestep_embedding(timesteps, self.dim_t))
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/container.py", line 217, in forward
    input = module(input)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py", line 116, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

ERROR :     Your simulation crashed :(. This could be because of several reasons. The most common are:
	 > Sometimes, issues in the simulation code itself can cause crashes. It's always a good idea to double-check your code for any potential bugs or inconsistencies that might be contributing to the problem. For example:
		 - You might be using a class attribute in your clients that hasn't been defined.
		 - There could be an incorrect method call to a 3rd party library (e.g., PyTorch).
		 - The return types of methods in your clients/strategies might be incorrect.
	 > Your system couldn't fit a single VirtualClient: try lowering `client_resources`.
	 > All the actors in your pool crashed. This could be because:
		 - You clients hit an out-of-memory (OOM) error and actors couldn't recover from it. Try launching your simulation with more generous `client_resources` setting (i.e. it seems {'num_cpus': 1, 'num_gpus': 0.0} is not enough for your run). Use fewer concurrent actors.
		 - You were running a multi-node simulation and all worker nodes disconnected. The head node might still be alive but cannot accommodate any actor with resources: {'num_cpus': 1, 'num_gpus': 0.0}.
Take a look at the Flower simulation examples for guidance <https://flower.ai/docs/framework/how-to-run-simulations.html>.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/flwr/simulation/app.py](https://localhost:8080/#) in start_simulation(client_fn, num_clients, clients_ids, client_resources, server, config, strategy, client_manager, ray_init_args, keep_initialised, actor_type, actor_kwargs, actor_scheduling)
    322         # Start training
--> 323         hist = run_fl(
    324             server=initialized_server,

16 frames
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/flwr/simulation/app.py](https://localhost:8080/#) in start_simulation(client_fn, num_clients, clients_ids, client_resources, server, config, strategy, client_manager, ray_init_args, keep_initialised, actor_type, actor_kwargs, actor_scheduling)
    357             client_resources,
    358         )
--> 359         raise RuntimeError("Simulation crashed.") from ex
    360
    361     finally:

RuntimeError: Simulation crashed.

It appears there's still a bug in the code.
The Client is not executing the model training process via the fit function. I've checked the implementation and configuration, but the issue persists.

Same Question.
Can't Run it on Colab.

I met the similar problem,
INFO :      configure_fit: strategy sampled 5 clients (out of 5)
INFO :      aggregate_fit: received 0 results and 5 failures
INFO :      configure_evaluate: strategy sampled 5 clients (out of 5)
INFO :      aggregate_evaluate: received 0 results and 5 failures

Hi there,
Sorry for the late reply. The reason for the issue was the recent flower library upgrades which seem to have poor backward compatibility with the version we used. I have updated the code so that it runs on the most recent flower version. In addition, I have added the requirement.txt with the current versions of all modules.
Hope that helps.


Add connection- and readtimeout configuration to connectiondata

The E3DCConnectionData object should have a connection and a readTimeout property.
The values are taken into account accordingly in the communication with the home power station

Implemented in 0.6.0


Added redirects for Passenger

When serving this application out through Passenger, certain actions
would lead to a HTTP 500 error. After inspecting the Rack::Request.env
attribute, it appears that request_path is missing and/or not set when
using Passenger. And since this value is used to redirect the user in
certain cases, Rack was throwing an error because it was nil. The fix
was to statically set the redirect variable for certain actions.

Thanks the report and the fix!
There is still a case where that error can happen (see def carefully in helpers.rb). I'm on mobile currently but I'll pull in this request and add a fix for that as well.

Hi @delano,
I saw that, but I couldn't trigger the same behaviour I was seeing as when I was trying to submit a blank secret for example. If I try to hit a fake page I do get properly redirected to the "Page not found" page, so I'm not sure if def carefully needs to be changed.

This probably can be closed; looks like you've fixed it in your recent releases :+1:
I haven't tested it, but will try to update sometime this month/early May.


File search always shows up empty

I have installed fzf, fd and bash. I also have ripgrep installed and I had the jethrokuan plugin but uninstalled with fisher rm before installing this one.
The commands all work except for the search files which always shows up as 0/0. The interface loads but it doesn't have any files to show.
I am using pop!_OS 20.04 and alacritty as my terminal emulator

Realised fd needs to be aliased to fdfind. Might be worth noting that it only works with fdfind as linked, or making uses it uses the correct fd

Hi Fraser! I'm confused. I don't have fd aliased to fdfind. How did you realize you needed to alias it to fdfind?

Hi Fraser! I'm confused. I don't have fd aliased to fdfind. How did you realize you needed to alias it to fdfind?

I went to the fd github page linked to in your README and read the installation instructions more carefully. The Ubuntu section has a paragraph below the apt command saying:

Note that the binary is called fdfind as the binary name fd is already used by another package. It is recommended that you add an alias fd=fdfind to your shells initialization file, in order to use fd in the same way as in this documentation.

Thanks! I may create an FAQ and add it there.


Enforce HTTPS

I'd like to set up my cerebro instance on heroku and make it accessible only via HTTPS.
Plan is to use either request filter or custom http action that will check in config if https enforcing is enabled. For http requests with https enforcing flag enabled it should return 400 saying that HTTPS is required or simply redirect to https

@pdziok hey there,
I'm not 100% sure I understood what you need/want.
cerebro is at the end a play application, and as such, can be configured in the same way. Could you check https://www.playframework.com/documentation/2.5.x/ConfiguringHttps#ssl-certificates to see if this would cover your needs?

Alternatively, you could set up nginx as a local proxy with SSL (which I'd recommend over exposing cerebro directly)

Hello, I successfully configured Cerebro over https only with play framework configuration inside application.conf:
play.server.https.keyStore.path = "/path/to/jks" play.server.https.keyStore.password = "pass" http.port = "disabled" https.port = "9000"


[FLINK-18905][task] Allow SourceOperator chaining with MultipleInputStreamTask

This PR adds a possibility to chain SourceOperator with MultipleInputStreamTask. Currently checkpointing, watermarks or metrics are not tested (part of another ticket).
Brief change log
Please check individual commit messages.
Verifying this change
This change added MultipleInputStreamTaskChainedSourcesTest for the new feature.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Thank you for the review @rkhachatryan, I've updated the PR. Could you take another look?

Done, @rkhachatryan can you take a look again?

Thanks, I've extended MultipleInputStreamTaskChainedSourcesTest#testLifeCycleOrder as we discussed offline. Will merge with green azure.


[soil] Delete the port

Unfortunately, the website no longer exists. Hopefully lonesock is okay...
cc @BillyONeal merge this one

I think last time we had a case like this we put it in ci.baseline.txt for ~6 months rather than outright removal?

https://github.com/microsoft/vcpkg/blob/7ad163e476f6ad9db9b6508d52f8f403b85c80d0/scripts/ci.baseline.txt#L1857

Hmm, okay.

Then maybe https://github.com/microsoft/vcpkg/pull/13000 should be merged first.

Applied https://github.com/microsoft/vcpkg/pull/13000 instead, thanks for the suggestion Nicole!


MOXD1 potentially incorrectly associated with HMR_4740

Description of the issue:
A Metabolic Atlas user noted that the HMR_4740 reaction:
1-piperideine-6-carboxylate[c] + H+[c] + H2O[c] <=> 2-aminoadipate 6-semialdehyde[c]
is associated with the MOXD1 gene, despite the gene encoding a monooxygenase with no known substrate. Furthermore, the references associated with the reaction did not contain any evidence of such an association.
After some investigation, it was found that this gene-reaction association originated from the EHMN model, yet no evidence supporting MOXD1 involvement is provided. Therefore, it is likely that this gene is incorrectly associated with the reaction and should be removed, but it should first be investigated further.
I hereby confirm that I have:

[X] Checked that a similar issue does not exist already

I still have not been able to find any evidence supporting the involvement of MOXD1 in this reaction (HMR_4740).
Interestingly, there also doesn't seem to be evidence for the other gene associated with this reaction, DBH. In fact, this reaction is reported as spontaneous (KEGG, MetaCyc), suggesting that this reaction should be associated with neither MOXD1 nor DBH.
I therefore plan to delete the grRule (MOXD1 or DBH) for reaction HMR_4740.

Gene association removed in #215.


Verify that requested CRDs are generated

controller-gen can complete successfully without generating the files
we expect, e.g. if the CRD type definitions are moved out of the
execution scope. This results in failure further down the build;
instead of waiting for that, this patch adds explicit checks for the
existence of the CRD we expect, and fails if it's missing.
Signed-off-by: Stephen Kitt<EMAIL_ADDRESS>

So this is catching an existing issue?
test -f deploy/crds/submariner.io_servicediscoveries.yaml
make: *** [Makefile:161: deploy/crds/submariner.io_servicediscoveries.yaml] Error 1


remove git2go dependency

Describe your changes
Remove git2go dependency and replace functionalities with go-git.
Screenshots - If Any (Optional)
This PR fixes:

Resolved #

Checklist before requesting a review

[ ] My code follows the style guidelines of this project
[ ] I have commented on my code, particularly in hard-to-understand areas
[ ] I have performed a self-review of my code
[ ] If it is a core feature, I have added thorough tests.
[ ] New and existing unit tests pass locally with my changes

Please open the PR against the dev branch (Unless the PR contains only documentation changes)

I still need to edit README

cc: @amirmalka

Don't merge it as is, we have perf regression.

I assume this fixes #832.

Will not be merged as explained here.


[BUG] LightRag - LLM call error

Description
Hi,
I am getting error when adding documents to LightRag index, it is a large file and gets broken down into 250 chunks of 1200 token each. However LLM call returned an error when handling Entity/Relationship extraction at chunk 197. Because of that, the indexing fail and I had to reindex, then it failed again at chunk 155. See error below.

Indexing [1/1]: document sample.txt
=> Converting document sample.txt to text
=> Converted document sample.txt to text
=> [document sample.txt] Processed 1 chunks
=> Finished indexing document sample.txt
[GraphRAG] Creating index... This can take a long time.
[GraphRAG] Indexed 0 / 1 documents.
Error: Error code: 500 - {'type': 'error', 'error': {'type': 'api_error', 'message': 'Internal server error'}}

Log info

Extracting entities from chunks:  80%|█████████████████████████████████▋        | 197/246 [1:17:33<19:17, 23.62s/chunk]
INFO:lightrag:Writing graph with 0 nodes, 0 edges

I have seen the exact same issue when using LightRag directly, and the solution was to add retry in front of the llm_model_func.
@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=4, max=10),
    retry=retry_if_exception_type((RateLimitError, APIConnectionError, Timeout, APIError)),
)

Can this retry be added to the Kotaemon? I am wasting a lot of Anthropic tokens on retrying indexing the same documents :)
Reproduction steps
1. Go to 'Files'
2. Click on 'LightRag'
3. Upload a large documents.

Screenshots
![DESCRIPTION](LINK.png)

Logs
Extracting entities from chunks:  80%|█████████████████████████████████▋        | 197/246 [1:17:33<19:17, 23.62s/chunk]
INFO:lightrag:Writing graph with 0 nodes, 0 edges

Browsers
No response
OS
No response
Additional information
No response

Hi, thanks for the report this seem to be a valid problem. Can you help to create a PR to include this fix?


[fix#107] new eslint parser to support typescript

Fixes #107
Describe the changes you have made in this PR -
replaced eslint-plugin-vue with vue-eslint-parser to support .ts files
other changes:
bumped version of @fortawesome/fontawesome-free, eslint, typescript, vue-tsc
converted arrayHelpers.js to arrayHelpers.ts as a proof of work
Screenshots of the changes (If any) -
Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

@vedant-jain03 @devartstar Can you review this PR?


ARROW-5309: [Python] clarify that Schema.append returns new object

https://issues.apache.org/jira/browse/ARROW-5309. I didn't find any other append method in the Python API (except for Table.append_column, which is already clear about returning a new table).

The failure was a cancelled build on ursabot.


LIVERY REQUEST: Philippine Airlines Boeing 777-300ER (OneWorld Livery)

Can You Resized It Please Thanks!


Resolve hosting concerns

We need to hash out how/where to host this file so it's not being served from vigesharing-is-vigecaring/awavering and it's being appropriately gzipped/cached. A dev might be needed to get us up and running on something; they know more about what accounts we have access to/would be best.

cc @ltk

Right now we're on S3 / CloudFront. @brianjlandau had said that https://www.jsdelivr.com/ might be worth looking at.

https://www.jsdelivr.com/ looks great :+1:

Did we get this? I thought we got this.

nevermind me, literally talking about this now.

For the time being we are sticking with S3 + CloudFront.


Exclusion doesn't work

Having an exclusion array like:
["*.html", "assets/fonts/**"]

will only exclude *.html, but not the folder fonts. In other grunt tasks this will work.
What is happening here?

The exclusions are implemented by using grunt's file.isMatch which actually uses glob. This would be the best place to figure out what and why is causing an issue.

@zonak Can you reproduce the issue? I don't know where to start as this exclusion string "assets/fonts/**" works in other grunt tasks, where glob is used too.

@zonak Bump

I just tested it and it works as expected for me. I did a similar configuration where I had both files and subfolders. Everything was excluded properly.


about and blog have hard-coded lower-case capitalization

Awesome theme! There seems to be an issue with hard-coded casing for the about and blog sections.
Describe the bug
I want to show my About and Blog pages with upper-case in the navbar. If I want to capitalize projects, I got to  _pages/projects.md and set title: Projects and the navbar shows this in uppercase. So I would think I could go to _pages/about.md and set title: About  to show About in upper case on the nav bar. But the casing is hard-coded in _includes/header.html so you cannot change the case, without changing _includes/header.html.
To Reproduce
Steps to reproduce the behavior:

go to _pages/about.md and set title: About

Expected behavior
The navbar should show the about section with the first letter capitalized (i.e. About)
System (please complete the following information):

OS: Max OS
Chrome 86.0.4240.198

Additional context
Here is where the names are hard-coded with lower case
https://github.com/alshedivat/al-folio/blob/master/_includes/header.html#L28
https://github.com/alshedivat/al-folio/blob/master/_includes/header.html#L38

Do you think that this should be a preference in config or hardcode to have sentence case @alshedivat?

Is this issue resolved? I am seeing the same behavior for project titles on the projects page too I am not to be get capitalized letters
To reproduce

In 1_project.md change the title to "Project Another Upper Case 1"
Expected Behaviour:
The title has capitalized letters, however it doesn't appear to be capitalised.

Also the same happens with navbar

Hi, You can modify the following line in _includes/projects.html:

{{ project.title }}

to simply

{{ project.title }}

and it seems to fix this.


59 auto linking broken

This PR is addressing #59 and

fixes the linking of dynamically created agents
adds (better) warning messages if the user provided invalid configurations
changes the attribute type of external_ids from dict to list

Furthermore, it

fixes an inconsistency that the trace_file is read relatively from the location of the GeneratorConfig like all other file paths

Please find a complete configuration for testing purposes attached.
testing_configuration.zip

@dlr-cjs: all issues resolved. please also find an updated testing configuration in the PR header description.

@dlr-cjs: changed external_ids to dict and allow list of str/int values, improved pattern matching to be more robust
please review


Service Feasibility - Progress on Export to CSV

When clicking Export to CSV on large datasets(3100 businesses), there is nothing showing that user that they need to wait.  The export took around 1 minute for me and I was not sure if it was working on not

@MikeMillerGIS We will update the code to display a loading indicator while exporting to CSV and will include the change in next release.

verified