nixos/jitsi-meet: init

Jitsi Meet video conferencing setup following these instructions (similar to the Debian quick install).
The java apps are not built from source but repackaged prebuilt JARs. I'm at my wit's end wrt building them from source so I'd like to ask someone more knowledgeable in maven to help me with this, potentially in another PR. (Edit: got source build mostly working.)
Feel free to test here: https://meet.b42.cz/
How to test
If you don't mind switching your system to the snapshot of NixOS Unstable this PR is based on, you can simply pass -I nixpkgs=https://github.com/otevrenamesta/nixpkgs/archive/jitsi-meet.tar.gz to nixos-rebuild or nixops.
If you want to only import the module without the rest of the OS, for example on 19.09 install, you can use the module from NUR:
imports = let
  nur-no-pkgs = import (builtins.fetchTarball "https://github.com/nix-community/NUR/archive/master.tar.gz") {};
in [
  nur-no-pkgs.repos.mmilata.modules.jitsi-meet
];

Example configuration:
services.jitsi-meet = {
  enable = true;
  hostName = "meet.example.org";
  videobridge.openFirewall = true;
};

services.nginx.virtualHosts.${config.services.jitsi-meet.hostName} = {
  enableACME = true;
  forceSSL = true;
};

networking.firewall.allowedTCPPorts = [ 80 443 ];

Motivation for this change
#82805
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg test jitsi-meet

@mmilata Wow, superamazing! Huge thanks! I'll try to test this today or during the weekend at least. Very excited. :slightly_smiling_face:
I suppose it's ok to build them from binary deb packages for now, but it'd be good to have "building from source" as todos for these packages?

@jluttine yeah that would be ideal, I've tried mavenix and mvn2nix and none worked for me, I guess I'd need advice from someone more experienced with Java/Maven. Though there seems to be quite a lot of packages in nixpkgs that just download archive with .jar like this.
Building the webapp through node2nix errors out on some missing webpack plugin, probably has simple solution I missed.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nixos-nixpkgs-nix-discord-for-voice-communication/5845/40

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/package-services-for-schools-covidsupport/6335/7

I tested this on localhost and seems to work! Just one minor issue/question, but it could be related to running it under localhost: I cannot access the front page of Jitsi Meet. Whenever I go to https://localhost, it just forwards me automatically to some room like https://localhost/HappyElephantsDanceRemorsefully. Is this on purpose, a bug or just my localhost setup issue?
This is how I configured:
    services.jitsi-meet = {
      enable = true;
      hostname = "localhost";
    };

I was expecting a similar front page as in https://meet.jit.si/ or is that just some custom configuration of their site?

@jluttine it's a feature, you can turn it off with services.jitsi-meet.config.enableWelcomePage = true;

@mmilata Ah, nice! Perhaps that would be a good example in the services.jitsi-meet.config option?

Thanks for review guys, pushed version that addresses some comments:

store secrets in /var/lib/jitsi-meet
use enableWelcomePage = true in default config, add it to examples
enableFirewall now uses ports from cfg.videobridge.config
mkDefault for nginx and prosody configuration

PTAL
Gonna attempt to build jitsi-meet from source again now.

Force-pushed:

jitsi-meet (the webapp) is now built (copied) from release tarball (which I stumbled into after fighting node2nix for half a day)

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/package-services-for-schools-covidsupport/6335/1

I would really like to use this as soon this is available.
Is there a way to use it as it is now?
Im a software developer myself so at least i could report back problems i encounter, but i have to say i am a real dummy when it gets to nixos.

@omgwtflaserguns if you don't mind running the version of NixOS Unstable this PR is based on, you can simply do nixos-rebuild -I nixpkgs=https://github.com/otevrenamesta/nixpkgs/archive/jitsi-meet.tar.gz switch using similar configuration as in nixos/tests/jitsi-meet.nix. nixops create can also accept -I argument like this.
If you want to run this on the stable version then I'm not sure there's easier way than checking out this branch, editing the module to replace e.g. pkgs.jicofo with pkgs.callPackage path/to/jicofo.nix {} and then putting the module in your imports = [ ... ];. Later today I will look into preparing overlay - that would make this significantly easier.

Pushed:

do not enableACME by default
rename hostname to hostName (networking.hostName seems authoritative enough)
add nginx.enable and prosody.enable options to allow turning off nginx/prosody integration & for the sake of documentation

@omgwtflaserguns I've uploaded the module to NUR so it can be tested more easily. See the updated PR description for instructions.

I am already on unstable so thats fine.
So i added the config from the first Post, and made a rebuild switch with your command.
See the result for yourself: https://meet.kurast.de/
:)
Thanks.

Will this be backported to 19.09?

@jacereda 20.03 release is planned for in 2 days, so a better question is if it will be backported to that. and i see no reason why not since it's a new package that don't affect others.

I'd love to see this package and module in upstream!

Pushed:

move journal-compatible logging config from module to packages because jitsi-videobridge2 needs different one

I've been able to build the Java apps from sources but haven't tested them very thoroughly yet. Upstream now considers videobridge2 stable, trying to find out what are the major differences.

I wanted to test this, but it seems to be conflicting with the ejabberd instance I have running on the same server. Is there a possibility to change the port of prosody?

@Lassulus would it normally make sense to run 2 jabber servers? maybe you just change the port in your configuration.

@davidak
oh I guess it would normally not make much sense :D
I basically wanted to keep my normal jabber server (ejabberd) running on the default port and setting prosody to some other port, but I couldn't find an easy way to do that.
configuring ejabberd to support jitsi also seemed a little bit complex for now.

would be cool if prosody could run on a socket so it wouldn't conflict with other jabber servers on the same machine.

that would be good.

great work.
it would be nice to have the interface_config.js configurable as well

@Lassulus @davidak changing ports might be theoretically possible using services.prosody.extraConfig and services.jitsi-meet.jicofo.config, this diagram might help. If you want to run this in a container you might also need to configure videobridge to run behind NAT. I'm not sure many users will want to run this on a system with another xmpp server, if anything I'd expect they'd want to integrate jitsi-meet into a prosody instance used also for instant messaging.
@betaboon There's small complication that interface_config.js is not a JSON but javascript-mostly-containing-JSON. In order to be able to override defaults we'd either have to work around this or keep copy of the default settings in the module which is probably not a good idea. I'll look into it, meanwhile you can replace the file nginx serves as interface_config.js like this.

@mmilata I'm looking at the upstream interface_config.js and as far as I can tell it's exactly as "javascript-containing-JSON" as config.js itself is. Is your concern just that it defines a lot more variables?
In that case, the only sensible way might be a dirty trick with custom JS and some creative use of Object.assign()...

@ArdaXi yeah, that's pretty much my main concern, config.js seems to work fine when we generate minimal one from scratch, I don't expect interface_config.js to be the same. Anyway, thanks for the feedback, I'll look into this matter today.

Cool. I did open an issue on their end as you can see because it seems like this should ideally be fixed upstream, but looking at their bug tracker I can sort of imagine they're a little bit swamped right now…
In the mean time, one thing that might make sense to add support for is authentication for room creation. The method of doing so is documented in the Jicofo repo.
I'm currently trying to set this up by myself (mainly to test) but I think it requires an mkForce on services.prosody.virtualHosts.${cfg.hostName}.extraConfig which is why I think it might be nice to have it supported in the main module.
No worries if you don't want to do this now, I'll gladly contribute the code myself, I'm just curious what you (and everyone else) think about this.

Pushed:

add interfaceConfig option
the services.jitsi-meet.config now overrides defaults obtained from the package (as does interfaceConfig)

@ArdaXi @betaboon It's pretty ugly (Node.js script with bunch of evals) but it's probably not as bad as using sed:D Please let me know if it works for you.
Regarding authentication I'd prefer to add this in subsequent PR to improve the chance of this one being merged in finite time:) Otherwise I'm not against the module including it.

Pushed:

update to current stable versions, upstream now uses jitsi-videobridge2 which is a major update - please test!
videobridge and jicofo no longer use native libraries, only JARs so running on ARM is possible but probably not a good idea
module now sets net.core.rmem_max and net.core.netdev_max_backlog sysctls

works great.
I got basic authentication working as well.
the relevant configuration in addition to default config:
  services.jitsi-meet.config.hosts.anonymous = "guests.${hostName}";
  # not sure if this is required
  services.jitsi-meet.config.enableUserRolesBasedOnToken = true;

  services.jitsi-meet.jicofo.config = {
    "org.jitsi.jicofo.auth.URL" = "XMPP:${hostName}";
  };

  services.prosody.virtualHosts.${hostName} = {
    enabled = true;
    domain = hostName;
    extraConfig = ''
      authentication = "internal_plain"
      c2s_require_encryption = false
      admins = {<EMAIL_ADDRESS>}
      Component "conference.${hostName}" "muc"
        storage = "memory"
      Component "jitsi-videobridge.${hostName}"
        component_secret = os.getenv("VIDEOBRIDGE_COMPONENT_SECRET")
    '';
  };

  services.prosody.virtualHosts."guests.${hostName}" = {
    enabled = true;
    domain = "guests.${hostName}";
    extraConfig = ''
      authentication = "anonymous"
      c2s_require_encryption = false
    '';
  };

@betaboon You'll want services.jitsi-meet.config.hosts.anonymousdomain rather than services.jitsi-meet.config.hosts.anonymous. That proper option is required for others to join the meeting without entering a password. Otherwise, that configuration works for me as well.

Is there any good guide on how authentication works with Jitsi? I'm fairly new to running an instance of this, and all the components seem pretty daunting.
That being said, this works great. I like the approach to configuration. Maybe we can get this PR into NixOS 20.03?

@numinit this guide explains how to setup a simple authentication scheme: https://github.com/jitsi/jicofo#secure-domain
my comment above demonstrates how to configure it in nixos based on the module in this PR

Specifically, authentication is handled by Prosody. As I understand it, a Jitsi call is based on an XMPP MUC (multi-user-conference) inside the main Prosody host. With the setup above, guests can join an existing MUC without a password, but cannot create new ones.
It would also be possible (but a bunch more complicated) to configure Prosody to talk to, say, LDAP.

I have had success with deploying it on Amazon EC2 by modifying the NixOps  tutorial https://github.com/KoviRobi/nixops-tutorial/blob/jitsi-example/jitsi-deployment.nix

@betaboon @ArdaXi I tried to replicate your configuration. It works fine up until Jitsi requests the meeting host to log in. However, the login hangs indefinitely.
I noticed that I cannot create users for my domain (let's use jitsi-meet.example.com for demonstration) with the command:
prosodyctl register <username> jitsi-meet.example.com <password>

This command fails with Error: Account creation/modification not supported.. Instead, I was able to create accounts with the subdomain auth.jitsi-meet.example.com.
Is that the expected behavior, or am I missing something?

@LutzCle Assuming you have authentication = "internal_plain" in services.prosody.virtualHosts.<hostName>.extraConfig, you should be able to run prosodyctl register <username> ${services.prosody.virtualHosts.<hostName>.domain} <password>
That error usually occurs when the Prosody host is not set up with authentication = "internal_plain" yet.

@ArdaXi Thanks, you are right. After manually setting authentication = "internal_plain" in /etc/prosody/prosody.cfg.lua`, adding a user and logging in works.
However, I also found the reason why it wasn't set in the first place. Literally copying betaboon's configuration results in the extraConfig options being appended to the extraConfig options set in the jitsi-meet.nix from this pull-request. Prosody then seems to select the first authentication option in the config file.
For me, this is not the expected behavior. In this particular case, I would expect an override of extraConfig instead of an append. Is there an easy way to achieve that?

@LutzCle That's because the standard behaviour in Nix for types.lines is to append multiple values if they're defined multiple times.
If you want to override this, you can do: services.prosody.virtualHosts.<hostName>.extraConfig = lib.mkForce "<options here>".
The mkForce will ensure the value will be overridden, not merged.

@ArdaXi pkgs.lib.mkForce works perfectly! Thank you for your help!

Pushed:

videobridge now connects to prosody as a user, not a component - this is the recommended setup for jvb2 and Debian package also now does it
switch to jvb2's new configuration format - services.jitsi-meet.videobridge.config is now attrs instead of attrsOf str
update to latest stable versions, adopt version numbers from Debian packages
rebase on top of current NixOS Unstable
move jitsi-meet package under pkgs/servers/web-apps and module under modules/services/web-apps

If you are upgrading from previous iteration of this PR please delete /var/lib/jitsi-meet/secrets and run systemctl restart jitsi-meet-init-secrets.
Apologies for breaking changes but I wanted to switch to the new configuration format before this gets merged. The old one is probably going to be supported for quite a long time but I suppose it's better to switch now when just a few people use the module.
Hopefully I won't make much more changes to the module so please review. I'll start pestering people to merge this when 20.03 is released.

@mmilata it seems like some settings are not yet available within the new config-format, thus i was required to write a /etc/jitsi/videobridge/sip-communicator.properties.
the settings in question (for NAT):

org.ice4j.ice.harvest.NAT_HARVESTER_LOCAL_ADDRESS
org.ice4j.ice.harvest.NAT_HARVESTER_PUBLIC_ADDRESS

i am not sure if they are just not available yet or if i just didn't find their new-config-equivalent in the documentation.
maybe it is a good idea to still have the sip-communicator.properties-file as a restartTrigger?

@betaboon yeah I think these two settings are not configurable through the new config, however you should be able to use services.jitsi-meet.videobridge.extraProperties to set them

@flokli thank you for your comments! I've tried to address them, PTAL.

@betaboon yeah I think these two settings are not configurable through the new config, however you should be able to use services.jitsi-meet.videobridge.extraProperties to set them

i can confirm this works :)

This MR comment stream is pretty long, but I can confirm that this works.
It would be nice if the jitsi p2p stuff had a clear path to make it work, but I think this MR is good to go and awesome!
If anyone wants to get their jitsi instance to do p2p, here is what I did to make it this work, as there is a lot of moving parts. But it was possible to get it working without any modifications to this MR.
  security.acme = {
    email =<EMAIL_ADDRESS>    acceptTerms = true;
    certs = {
      ${config.services.coturn.realm} = {
        postRun = "systemctl restart coturn.service";
        user = "turnserver";
        group = "turnserver";
        webroot = "/var/lib/acme/acme-challenges";
      };
    };
  };

  services.nginx = {
    enable = true;
    recommendedProxySettings = true;
    virtualHosts."default.example.com" = {
      default = true;
      root = "/var/lib/acme/acme-challenges";
    };
    virtualHosts."meet.example.com" = {
      enableACME = true;
      forceSSL = true;
    };
  };

  services.coturn = rec {
    enable = true;
    no-cli = true;
    no-tcp-relay = true;
    min-port = 49000;
    max-port = 50000;
    use-auth-secret = true;
    static-auth-secret = "REDACTED";
    realm = "turn.example.com";
    cert = "${config.security.acme.certs.${realm}.directory}/full.pem";
    pkey = "${config.security.acme.certs.${realm}.directory}/key.pem";
    extraConfig = ''
      # for debugging
      verbose
      # ban private IP ranges
      denied-peer-ip=<IP_ADDRESS>-<IP_ADDRESS>
      denied-peer-ip=<IP_ADDRESS>-<IP_ADDRESS>
      denied-peer-ip=<IP_ADDRESS>-<IP_ADDRESS>
      denied-peer-ip=<IP_ADDRESS>-<IP_ADDRESS>
      denied-peer-ip=<IP_ADDRESS>-<IP_ADDRESS>
      denied-peer-ip=<IP_ADDRESS>-<IP_ADDRESS>
    '';
  };

  services.prosody.extraConfig = ''
    turncredentials_secret = "${config.services.coturn.static-auth-secret}";
    turncredentials_port = 3478;
    turncredentials_ttl = 3600;
    turncredentials = {
      { type = "stun", host = "${config.services.coturn.realm}" },
      { type = "turn", host = "${config.services.coturn.realm}", port = 5349},
      { type = "turns", host = "${config.services.coturn.realm}", port = 5349, transport = "tcp" }
    }
  '';
  services.prosody.extraPluginPaths = [ "/etc/prosody/plugins" ];
  services.prosody.extraModules = [ "turncredentials" ];

  services.jitsi-meet = {
    enable = true;
    hostName = "meet.example.com";
    videobridge.openFirewall = true;
    config = {
      useStunTurn = true;
      p2p = {
        enabled = true;
        preferH264 = true;
        useStunTurn = true; #, // Using Turn for p2p connections
        stunServers = [
          { urls = "stun:${config.services.coturn.realm}:3478"; }
          { urls = "stun:stun.l.google.com:19302"; }
          { urls = "stun:stun1.l.google.com:19302"; }
          { urls = "stun:stun2.l.google.com:19302"; }
        ];
      };
    };
  };

@georgyo Out of curiosity, why do you find using a TURN server preferable to using the videobridge? Jitsi should already fallback to the bridge if STUN doesn't work.

@georgyo Out of curiosity, why do you find using a TURN server preferable to using the videobridge? Jitsi should already fallback to the bridge if STUN doesn't work.

Yes, it is possible I don't 't need it. I did notice that jitsi does not default to using p2p for 1-on-1 connections. You must at least enable p2p and specify stun servers.
I started yak shaving. The same server I am running this on is also running matrix, which does require a TURN server. It seemed logical to use the same turn server for jitsi as matrix.
But since as soon as third party joins it must go though the video bridge anyway, you are right that TURN is not providing all that much value.
Still, googling for jitsi and p2p all heavily suggest that a turn server is good.

@georgyo Jitsi does default to P2P for 1-on-1 connections. Both that option and STUN servers are present in the default config which this module uses: https://github.com/jitsi/jitsi-meet/blob/master/config.js#L337
What you've probably run into is that P2P is impossible if both parties are behind a symmetric NAT. As part of ICE, the client detects that a direct P2P communication is impossible and will fall back to using the videobridge, as it would for a call with more than two people. If you have a TURN relay available, it will use that instead, before declaring P2P a failure.
A TURN server does essentially the same thing a videobridge does: rather than connecting two clients together, all their traffic passes through the TURN relay. I don't really see the benefit of running a TURN server (with the security/abuse risks that implies) while the bridge performs exactly the same function.
The only thing I'd note is that presently, a TURN relay will preserve E2EE which the bridge will not. However, that's being worked on too: https://jitsi.org/news/e2ee/ and since WebRTC does not provide authentication, the security benefit is marginal anyway.
All of which is to say I don't think it's a very worthwhile idea to add coturn integration into the jitsi module, but if you still think it's something you want it could be added (after this PR) with a warning in the docs.

@georgyo thank you for the configuration sample! I think having an option to enable coturn could be a nice feature - preferably in another PR though, this one is getting quite large already:/
FWIW the Debian installer also installs it. The Docker image doesn't, perhaps due to more involved network setup?

So… this might be a weird one, and might not fit into this PR (we're still hoping to get it merged ASAP, after all). But there are applications for having some values in (primarily) config.js be somewhat dynamic, rather than hard-coded. I'd considered the idea to have some server-side application handle the request for config.js, but then I realised it is a pure Javascript file, not a JSON file. What if it were possible to add custom JS to the end of the generated config.js file?
This is not entirely theoretical, either. As an example, I would like to be able to host Jitsi on multiple hostnames, and add something like config.bosh = "//" + window.location.hostname + "/http-bind" in order to avoid CORS issues.
Another potential use-case would be to set deploymentInfo if you have a setup with octo enabled, so as to detect which region the user is in and make sure it gets assigned to a bridge close to them.
I acknowledge this might be a little bit ridiculous, but at the same time I think it would be an okay solution. Making the config available to some other process to do this server-side seems a lot more complicated.

Thanks for this amazing MR :)
It worked great for me for 1-1 visioconfs, between a computer browser and another computer browser (firefox-chromium).
When trying to add an android app client to the visioconf it bugs though. When cellphone joins the meeting, video and sound are shutoff for all users. Only text chat works. It might be a more-than-2-users issue rather than a cellphone app issue though.
Surprisingly your instance https://meet.b42.cz/ worked well in this scenario :  2 computer browsers and a cellphone app.
Since I use an AWS EC2, it might be a security group issue (kind of AWS firewall). My allowed ports in security groups are 22, 80 and 443.
Scanning your ports show
22/tcp   open  ssh
80/tcp   open  http
443/tcp  open  https
8080/tcp open  http-proxy

Do I need this "http-proxy" service and a 8080 open port ?
Other fact, video on your instance is substantially faster than on mine. Maybe issues are caused by my weaker cpu.

@JosephLucas It's a bit more complicated than that. While you're with just two people, the connection is straight P2P, so it works. As soon as you have more than two, all the audio and video streams are passed through a component called the Jitsi Videobridge (jvb).
As you can see in the PR, you'll need to open up jvbConfig.videobridge.ice.tcp.port and jvbConfig.videobridge.ice.udp.port (TCP and UDP respectively.) The defaults are 4443/tcp and 10000/udp.
The jvb will also need to know about its own public IP. If your EC2 instance has a public IPv4 address directly attached to it, the jvb will discover it from the metadata service on its own. If not, it will have to be set, but the exact code required for that escapes me at the moment.

@ArdaXi it works, thanks !

Pushed:

replace phases with dontBuild = true in jitsi-meet derivation
rename passthru.tests test from jitsi-meet to single-host-smoke-test
in module, add mkDefault to nginx locations serving config files
rebase to master to fix conflict

@ArdaXi TBH I haven't looked into what's needed for setting up Octo (cascading videobridges) yet. Also not sure what would a reasonable configuration interface for what you propose look like. For now it's probably best to create config.js manually and then configure nginx to serve it instead of the generated one:
services.nginx.virtualHosts.${hostName}.locations."=/config.js".alias = yourConfig;

Added mkDefault there so you shouldn't need mkForce.

@mmilata I think that first it would help to have the videobridge split out into a separate module. The Jitsi team recommends (if you end up supporting more than a few conferences at a time) to run prosody/jicofo/nginx on one machine, and have the jvb run on a group of VMs where you can spin up more if one ends up using a certain amount of network bandwidth. This already helps because the jvb is the only really network-intensive bit of the setup.
My motivation for having some kind of extraJs attr or something would be to not have to reinvent the wheel with the config handling that's here already.
I recognise that this is certainly not needed to get this in right now. I would really rather see this merged ASAP. I would like to create a PR with what I have in mind for this afterwards.

@ArdaXi I'm hesitant on merging this before it's ready. There's a lot of flying components that are not yet fully fleshed out.
Once it's in nixpkgs, people might expect it to work and existing configurations not being broken. If people want to use this now, even before it's ready, they can just import the module code into their configuration.

@flokli I certainly didn't want to say "merge it before it's ready." The points I'm mentioning are, in my opinion, not things that would need to be addressed before it works. The implementation I had in mind would not break existing configurations.

For your information, I've been working on a prosody module revamp
lately. It exposes the mucs and the TLS config in a clean way. It will
probably make things easier for this PR.
https://github.com/NinjaTrappeur/nixpkgs/commit/aca7d71fba98c60d5db0d5436d641e9ff30b3f42
I'm doing that on my spare time: it's hard to give a ETA. That being
said, I re-wrote the nixos test yesterday night. This was the last major
thing to do, I'm pretty close to open a PR for that. It will probably
happen during this week (again, not a hard deadline, it might slip).
I agree with flokli regarding the self-signed certificates: we should
allow users to provision the XMPP endpoints with a ACME certificate,
that's a totally reasonable thing to expect. We have a whole
infrastructure making easy to generate those in nixpkgs, we probably
want to leverage that here.

I'm a bit worried about the long term maintainance of this module.
Writing a nixos VM test would make maintainance much easier. That
being said, it's probably easier said than done. I'm not sure how to
proceed to test the actual webrtc conference :/
There are a lot of moving interconnected parts in this PR. I'm also
quite worried about the inter-modules weirdness we could end up
encountering in a real world server setup.
That being said, most of this complexity is coming from Jitsi itself.
I'm not sure this should live in nixpkgs.

@ArdaXi

I think that first it would help to have the videobridge split out into a separate module.

My plan is to add config.services.jitsi-meet.videobridge.enable and another option to configure connections to XMPP server(s). Was considering putting videobridge as well as jicofo under their own attributes under config.services, however the way it is now makes it easier to set up the single host configuration most people probably want. What do you think?
@NinjaTrappeur that looks good, I'll be happy to rebase this module on top of your changes!
Re ACME certs, the plan was to add the support in future PR but I guess it can fit in this one. Regarding tests, I have no clue how to test anything more than that the services start and web application is reachable:(
Btw I'm also working on this on my spare time, if this doesn't belong to nixpkgs then please let me know sooner rather than later.

@mmilata I don't mind having jicofo as part of the jitsi-meet module, but I think that the videobridge at least definitely deserves its own module. It's the one thing that you would definitely want to put on its own in an every day scenario.
I would implement that in the same way as config.services.jitsi-meet.nginx.enable, so if that's set to true it would enable config.services.jitsi-videobridge and put default settings in it. If that property defaults to true, setting up the single-host is still the same, while also making it easy to do it separately.
I don't really see a use-case for setting up just jicofo on a host, so I'm not sure I'd give that its own module, but I suppose it could be done.

Regarding tests, I have no clue how to test anything more than that
the services start and web application is reachable:(

Indeed, I don't think testing whether the webapp is reachable or not add
much value here. That being said, if my mental model around Jitsi is
correct (it might not be!), I understand that under the hood, it uses a
XMPP server as signaling server in charge of setting up a sip link
between two the computer and the media server.
I guess a good way to test this module would be to:

Testing the signaling setup by requesting a SIP setup using a XMPP
client.
Using a SIP softphone to setup the connection using the previously
fetched SIP parameters.

Maybe it would worth investigating jigasi
https://github.com/jitsi/jigasi. Accounding to its readme, it looks like
we could use it to implement point 1 defined above.
We could then use it to setup a softphone.
Again, I'm not a Jitsi expert, I'm not sure this is the right way to
go.

Btw I'm also working on this on my spare time, if this doesn't belong
to nixpkgs then please let me know sooner rather than later.

I feel like answering to this since my previous message was ending up
with "I'm not sure this should live in nixpkgs" before I edit that part
out.
I edited out that part for 3 main reasons:

You spent a lot of time working on this. I feel like this remark was
unecessary rude. I'm sorry if I made you feel bad, this remark was
out of place.
I wouldn't maintain this module as it is, that said, I won't use and.
Furthermore, since you created this PR and are the de-facto
maintainer, my opinion does not matter here.
I'm not a nixpkgs committer, I have no autority to make this call. On
top of that there is no clear policies about what should be
integrated in nixpkgs and whatnot.

I'll ping you and provide you with a new prosody config once my module gets
upstreamed (if it does at some point ;)).

In my opinion, this kind of NixOS services in nixpkgs are one of the main things that make NixOS so amazing in practice (at least for me). Also, if this wasn't part of nixpkgs but in NUR or elsewhere, it wouldn't receive this amount of extremely valuable reviews that are important for this kind of complex services. Just thought I'd say, really excited about this. But yeah, I don't really know about nixpkgs policies nor have any authority but I definitely hope this and similar services are and will be included in nixpkgs. Great work! :gem:

Jitsi definitely has some very weird quirks, which make building a NixOS module a bit more challenging.
IMHO, if the configuration structure has been battle-proofed to be composable enough to work with the various usecases, and there's sufficient testing infrastructure provided to ensure these don't break in the future, and people are actively maintaining and using it, I don't see a reason to not accept it.
I just want to avoid merging it prematurely. Changing some assumptions later on always comes with a lot of churn for both maintainers and users, so I'd rather prefer a module like this to have undergone some thorough review.

@NinjaTrappeur

I understand that under the hood, it uses a
XMPP server as signaling server in charge of setting up a sip link
between two the computer and the media server.

This is not true. It uses XMPP as a signaling server to set up a WebRTC link. Jigasi is a component that can also hook in a SIP connection, but unless you want to be able to dial in/out, SIP is not used. It's purely WebRTC, which could still be tested in theory.

great initiative!
is Jibri (for recording) also part of this PR?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nixcon-2020-status-update/6914/6

Pushed:

moved jitsi-videobridge into its own module services.jitsi-videobridge

setting services.jitsi-meet.videobridge.enable = true enables this service and configures it to connect to XMPP on localhost

made the videobridge HOCON config generation slightly saner (IMO)
added options for configuring NAT addresses
jicofo now skips certificate validation instead of fiddling with JKS, should be fine since we're connecting to localhost
added check for required environment variables to jitsi-meet-init-secrets
updated to latest stable versions

Feedback appreciated. Hoping to address the other outstanding issues in the near future.

@Zhen-hao Sorry, not in this PR.

Pushed:

Rebased on top of @NinjaTrappeur's #86067 which should make Jitsi Meet work a bit better with Prosody instance shared for other use.
Split off JiCoFo to a separate module. Even though 99 % of users will want to run it on the same host as Prosody, this has two advantages:

Less complexity in jitsi-meet.nix and coupling of the two modules through a defined set of options.
jitsi-meet.nix now acts purely as an "integration" module that configures other "service" modules. Should make it easier to integrate them in a different way if one desires.

Secrets for the "service" modules are now configured using the passwordFile pattern, i.e. you provide a path to file containing the secret. The "integration" module generates these files for you, unless you want to explicitly set videobridge password for connecting multiple videobridges to one Prosody instance. This change does not require any action if you're upgrading from previous version of this PR.
Self-signed prosody certificate is now generated using openssl instead of prosodyctl cert generate and it is now not needed to regenerate the cert. ACME integration is still on TODO list.
services.jitsi-meet.extraConfig option was added to allow injecting javascript for bridge selection logic (cc @ArdaXi)

According to https://github.com/jitsi/jitsi-meet/blob/master/doc/scalable-installation.md#jitsi-videobridge2 it should be no longer needed to configure IP addresses when running behind NAT - can someone confirm this please?

I was also trying to improve the test and it should in theory be possible to test this end-to-end, i.e. including the javascript that runs in browser by means of the enableOCR test option, v4l2loopback kernel module and xdo tool.
The module must be enabled in node configuration:
boot.extraModulePackages = with config.boot.kernelPackages; [ v4l2loopback ];

Then it's possible to create mock camera that will output static image with text:
modprobe v4l2loopback
convert -border 250 -font DejaVu-Serif -pointsize 50 -extent 640x480 label:'Hello Client1' /tmp/image.png
ffmpeg -loop 1 -re -i /tmp/image.png -f v4l2 -vcodec rawvideo -pix_fmt yuv420p /dev/video0

This is where I got stuck:

I can't figure out how to import CA certificate into Firefox from the command line (without excessive yak-shaving). This is needed because camera can only be used in secure context i.e. over https.
Chromium doesn't detect the mock camera.

Any ideas?

I think there is a v4l2loopback option that helps with Chromium. Look in v4l2loopback issue tracker or something. (Sorry, on mobile.)

To whom it may concern: I copied the relevant module/package files from this PR to my nur-packages repo so that you can more easily use this PR without patching your nixpkgs. Like this PR, it depends on prosody changes on nixpkgs-unstable.
There is also a sample config that works with recent updates of this PR.
[The nur-packages repo @mmilata had some problems.]

Regarding adding a certificate in firefox, I think the only way is to do something like
nix run nixpkgs.nssTools -c certutil -A -n "name" -t "TC,," -d sql:$HOME/.pki/nssdb -i name.pem

Though, it would be good if someone could figure how to point firefox at the system, it's still an open issue.

Is sombody still working on this?

@bachp I don't have the motivation to work on this any more, at least not anytime soon. Apologies to everyone. Feel free to reuse expressions from this PR.
IIRC the main thing that is missing is LetsEncrypt support for Prosody<->{jicofo,videobridge} certificates.
Here's OCR-enabled test of video functionality that doesn't work reliably.

Thanks for your work @mmilata, using a modified version of @kolloch nur I made jitsi work fine. I wish that this PR will soon be completed and merged.

Can we have a MVP (with some documentation to get the thing working) and have new features add as Pull Requests?
It would be fantastic to have jitisi as module in nixpkgs!

Thanks to everyone who contributed to this. Some of my friends and I use it since a while for socializing :smile:
I have one small improvement request: The nginx config is based off the manual install guide, while the Debian packages contain a completely different nginx config. The Debian config allows for "-" characters in room names, which is required when using the Matrix/Riot integration, because that generates room names with "-" characters. It would be great to change to a nginx config based on the Debian package.

I agree with makefu that feature / improvement requests should not keep this module from being included in nixpkgs. I'm generally interested in continuing this PR up to the point where it can be merged as a minimum viable product, but I'm not interested in maintaining the package and module long-term, so if anyone is up to that task feel free to contact me so we can make it work.

FYI: I started a second try with https://github.com/NixOS/nixpkgs/pull/92468


How to use permissions?

I do not understand how I should implement the global variable IMPORT_EXPORT_EXPORT_PERMISSION_CODE to avoid the "export" button to groups that do not have permissions. Can you give an example, please?

class Book(models.Model):
    id = models.AutoField(primary_key=True)
    name = models.CharField(max_length=60)

    class Meta:
       permissions = (('import', 'Can import'), ('export', 'Can export'))

Then run makemigrations and migrate
This will generate the permissions as per django documentation, but django-import-export isn't really working with anything but the default.

With custom permissions no problem.
When you set for example view permission, it means view_book permission. If you want to use custom permission, you need set for example IMPORT_EXPORT_EXPORT_PERMISSION_CODE = 'import' in your settings. And in your Bookmodel it will beimport_bookinstead ofimport` permission

Can someone share working code example?

For the permissions to work you need to do the following:

class Book(models.Model):
    id = models.AutoField(primary_key=True)
    name = models.CharField(max_length=60)

    class Meta:
       permissions = (('import', 'Can import'), ('export', 'Can export'))

Then run makemigrations and migrate This will generate the permissions as per django documentation, but django-import-export isn't really working with anything but the default.

After doing this you must go to the project settings and add the following environment variables:
IMPORT_EXPORT_IMPORT_PERMISSION_CODE = "import"
IMPORT_EXPORT_EXPORT_PERMISSION_CODE = "export"

Thanks, Carlos, for the code sample! I would like to add that the permission codename is not optional.
Does not work:
class VeryLongModelName(models.Model):
    …

    class Meta:
       permissions = [("import_shortname", "Import …")]

Does work:
class VeryLongModelName(models.Model):
    …

    class Meta:
       permissions = [("import_verylongmodelname", "Import …")]

Explanation:

https://github.com/django-import-export/django-import-export/blob/e5f5b3cb842b8a2336fc4a6927b2297223fc2a72/import_export/admin.py#L130
https://github.com/django/django/blob/8edaf07a28a3e3848ab1edbdcd2fdc25e6bd6015/django/contrib/auth/__init__.py#L253
https://stackoverflow.com/questions/64984409/understanding-model-permissions-in-django-3-1


ENH: Expose FFT filter dimension traits

Moves FFT filter image dimensions into existing trait objects in order
to decouple FFT factory overrides from fixed dimension lists. This will
allow other, external FFT implementations that may not be valid for
higher-dimension overrides to reuse the FFT factory class.
Future changes may replace the explicit dimension lists with CMake
variables to avoid registering unnecessary factories.
Motivation for these changes comes from developing accelerated FFT filter overrides in the ITKVkFFTBackend external module.

PR Checklist

[x] No API changes were made (or the changes have been approved)
[x] No major design changes were made (or the changes have been approved)

Refer to the ITK Software Guide for
further development details if necessary.

Looks like a good step forward.
It is a shame that the std::make_integer_sequence  starts at 0, so we can't directly use it. The results of it could have additional performed on each integer element with a variadic template to perform addition. I could write a class to do that if you think it would help with the final cmake parameters needed.

@blowekamp I haven't tried it yet but I think the simplest way would be to rely on the explicit list of dimensions specified by the user in ITK_WRAP_IMAGE_DIMS, or define a similar CMake variable to do the same. That way the logic for creating dimension lists would be handled in the CMake config step and the code you've already written (thanks!) for unpacking the integer_sequence would suffice for handling overrides at compile time.

I didn't dive deeply enough to understand these variadic templates ... apologies if this is a dumb question:
In
https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Filtering/FFT/include/itkFFTImageFilterFactory.h#L98
and
https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Filtering/FFT/include/itkFFTImageFilterFactory.h#L108
what is the advantage of having the likes of
const std::integer_sequence<unsigned int, D, ImageDimensions...> &

as an argument to the method?  Could the expression that appears as an argument be eliminated on line 98 making this a function of void?  On line #108, could the type that appears as an argument be placed with the other template parameters rather than as a function argument?
I am thinking that the usual arguments about partially templated methods do not apply.  Is it a matter of style / preference?

@Leengit It looks like there was discussion around this in #3065 regarding how the recursive procedure is terminated, to me this falls under an "if it ain't broke" scenario. If you and @blowekamp decide that a different implementation would be advantageous it may be best to track discussion in a dedicated issue.

how the recursive procedure is terminated

Of course.  That mine was a dumb question is now confirmed.  :-)

@Leengit Not a dumb question, thanks for double-checking on something that wasn't immediately clear. If/when ITK moves to C++17 one day in the future, we could replace the current recursive implementation with something like a fold expression to greatly improve readability.


Testing a hypothesis

The failing test (on TravisCI only):
./spec/controllers/users_controller/admin_user_spec.rb:110

with error:
1) UsersController admin user PUT #update with valid params and a new email assigns the requested user as @user
     Failure/Error: create_list :user, 3, office: create(:office)
     ActiveRecord::RecordInvalid:
       Validation failed: Name has already been taken
     # ./spec/controllers/users_controller/admin_user_spec.rb:15:in `block (3 levels) in <top (required)>'

Trying to create the users with 'create_list', but omitting the office
creation, to see how it fares.

There were the following issues with your Pull Request

Commit: 34bccaff5c6469a8bdb923bf4ce19c92ef35f09f
Your commit message body contains a line that is longer than 72 characters

Guidelines are available at https://github.com/ministryofjustice/fr-staffapp

This message was auto-generated by https://gitcop.com

@colinbruce can you have a look at this PR, so that we can have a chat when you're back on Monday?

LGTM.

Can one of the admins verify this patch?


*.Views.dll not in AppDomain.CurrentDomain.GetAssemblies()

Describe the bug
Creating a new .Net Core 2.2 MVC project.
Why *.Views.dll not in AppDomain.CurrentDomain.GetAssemblies()??
To Reproduce
Steps to reproduce the behavior:

Using this version of ASP.NET Core '...'
V 2.2
Run this code '....'
Using  AppDomain.CurrentDomain.GetAssemblies()
With these arguments '....'
See error

Expected behavior
I thought the ProjectNmae.View.dll should place in AppDomain.
Is this a misunderstanding??  ="{
Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
.NET Core SDK (reflecting any global.json):
Version:   2.2.103
Commit:    8edbc2570a
Runtime Environment:
OS Name:     Windows
OS Version:  10.0.17134
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\2.2.103\
Host (useful for support):
Version: 2.2.1
Commit:  878dd11e62
.NET Core SDKs installed:
2.1.202 [C:\Program Files\dotnet\sdk]
2.1.503 [C:\Program Files\dotnet\sdk]
2.2.103 [C:\Program Files\dotnet\sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Thanks for contacting us, @yuessir.
We load these generated View assemblies at runtime.


F:022 Caused through walletconnect.

Invalid Signature while trying to claim with walletconnect (Bridge Wallet)
To Reproduce

Connect wallet to connextscan with Bridge Wallet (https://www.mtpelerin.com/bridge-wallet) through WalletConnect
Try to claim and sign in Bridge Wallet

Additional context
I haven't checked with other wallets via WalletConnect. The user cannot import the seed phrase into the metamask, has to wait for the expiration date.

Hi, this is not a wallet connect issue, this is a wallet signature issue. Can you please try claiming from connextscan.io?

Considering transaction must had a conclusion in 3 days on it's creation, this issue should have been resolved.


How to edit max_query_size

I have 800 sigma rules i have to edit max_query_size in each file is there any way to edit

https://github.com/Yelp/elastalert/issues/3027

https://github.com/Yelp/elastalert/issues/3027

hey @nsano-rururu is this correct
tools/sigmac -t elastalert -c tools/config/generic/sysmon.yml -c sigmac/sigmac-config.yml --backend-option timestamp_field=etl_processed_time max_query_size=10 -o /root/sigma_rules/sigma_sysmon_win_advanced_ip_scanner.yml rules/windows/process_creation/win_advanced_ip_scanner.yml

hey @nsano-rururu is this correct
tools/sigmac -t elastalert -c tools/config/generic/sysmon.yml -c sigmac/sigmac-config.yml --backend-option timestamp_field=etl_processed_time max_query_size=10 -o /root/sigma_rules/sigma_sysmon_win_advanced_ip_scanner.yml rules/windows/process_creation/win_advanced_ip_scanner.yml

Error while running the above
usage: sigmac [-h] [--recurse] [--filter FILTER]
[--target {fireeye-helix,es-rule,kibana,grep,arcsight,arcsight-esm,elastalert-dsl,uberagent,es-qs,es-dsl,carbonblack,sumologic,sumologic-cse,graylog,stix,qradar,ala,fieldlist,logiq,sql,powershell,sqlite,sysmon,elastalert,xpack-watcher,limacharlie,qualys,splunk,csharp,ee-outliers,humio,splunkxml,sumologic-cse-rule,mdatp,netwitness,logpoint,kibana-ndjson,ala-rule,netwitness-epl,crowdstrike}]
[--lists] [--config CONFIG] [--output OUTPUT] [--print0]
[--backend-option BACKEND_OPTION]
[--backend-config BACKEND_CONFIG] [--backend-help BACKEND_HELP]
[--defer-abort] [--ignore-backend-errors] [--verbose] [--debug]
[inputs [inputs ...]]
sigmac: error: unrecognized arguments: rules/windows/process_creation/win_advanced_ip_scanner.yml

Error while running the above
usage: sigmac [-h] [--recurse] [--filter FILTER]
[--target {fireeye-helix,es-rule,kibana,grep,arcsight,arcsight-esm,elastalert-dsl,uberagent,es-qs,es-dsl,carbonblack,sumologic,sumologic-cse,graylog,stix,qradar,ala,fieldlist,logiq,sql,powershell,sqlite,sysmon,elastalert,xpack-watcher,limacharlie,qualys,splunk,csharp,ee-outliers,humio,splunkxml,sumologic-cse-rule,mdatp,netwitness,logpoint,kibana-ndjson,ala-rule,netwitness-epl,crowdstrike}]
[--lists] [--config CONFIG] [--output OUTPUT] [--print0]
[--backend-option BACKEND_OPTION]
[--backend-config BACKEND_CONFIG] [--backend-help BACKEND_HELP]
[--defer-abort] [--ignore-backend-errors] [--verbose] [--debug]
[inputs [inputs ...]]
sigmac: error: unrecognized arguments: rules/windows/process_creation/win_advanced_ip_scanner.yml

I don't know what sigmac is, and I don't normally use ElastAlert at all so I can't answer questions about the rules.

I don't know what sigmac is, and I don't normally use ElastAlert at all so I can't answer questions about the rules.

No one answers. Search for past issues.

No one answers. Search for past issues.

Yelp/elastalert is no longer maintained. Please use jertel elastalert. Questions to the discussion below
https://github.com/jertel/elastalert/discussions


Daemon user

Run the daemon as a separate user: (Redefine DIGCROSSD_USER in CMakeLists.txt, create a new user as a part of the install process.)

Fixed in 390daecc56ed07115db1815199451cca768d440b


Add unit tests for sock_addr and path_offset functions

Motivation
The sock_addr function dispatches to a fair amount of libc calls and has
some additional complexity that is really only explained by man pages. It does
not have any unit tests and was incorrectly calculating addrlen.
Solution
A new path_offset function has been separated out from sock_addr that
returns values dependent on the platform. It is a single function that can be
unit tested and now also used within the sys::unix::uds module.
Basic unit tests have also been added that check correct values for pathname
socket addresses and abstract socket addresses.
Signed-off-by: Kevin Leimkuhler<EMAIL_ADDRESS>

LGTM pending the CI fix.

@Thomasdezeeuw Looks like a spurious failure in CI. Tested locally on a Mac as well to double check.

@glelk These are unit tests and refactor I was talking about these functions benefitting from!


[MKL] Get/Set number of threads - should get/set for BLAS, not for the whole MKL

If MKL.jl (BLAS/LAPACK) is used with other packages providing access to others MKL features (FFT, VML, etc.) then a call to BLAS.set_num_threads(_) will set the maximum number of threads used by all this packages.
So if you need to manage threading yourself, e.g. for an embarrassingly parallelizable algo working on a lots of small matrix, then you would do a BLAS.set_num_threads(1). But at this point your FFT and VML calls also become single threaded.
MKL has a notion of "domain", see: mkl\include\mkl_types.h

define MKL_DOMAIN_ALL  0
define MKL_DOMAIN_BLAS  1
define MKL_DOMAIN_FFT 2
define  MKL_DOMAIN_VML  3
define MKL_DOMAIN_PARDISO  4

All service settings functions should work on the domain they belong.
See MKL support-functions
The correct call:
set_num_threads(numthreads::Int) =
    Bool(ccall((:MKL_Domain_Set_Num_Threads, MKL_jll.libmkl_rt),
    Cuint, (Cint,Cint), numthreads, MKL_DOMAIN_BLAS))

Since the method is BLAS.set_num_threads, not VML or FFT...
The current behavior interfere with VML, and as it is with the PR https://github.com/JuliaMath/IntelVectorMath.jl/pull/55.
I think one should special case MKL in
https://github.com/JuliaLinearAlgebra/libblastrampoline/blob/main/src/threading.c
(and maybe elsewhere) since it's more than a BLAS/LAPACK library.

Hmmm, I think I agree!  I would be happy to accept a PR making this change, to force MKL to use MKL_Domain_Set_Num_Threads instead of MKL_Set_Num_Threads.

You also need to use MKL_Domain_Get_Max_Threads and since the Domain value has to be passed, the internal APIs inside LBT need to be adjusted - I think they expect these APIs roughly to be similar across different BLAS, but now that will change. Should be straightforward to incorporate, but it appears to need a bit more work than just find and replace.

Yes, we need to essentially special-case this.

This is pretty difficult to do. It would be best to not special case this mechanism in libblastrampoline, and instead let the various BLAS packages provide this at the Julia level.


1.107

GAME_VERSION = "1.107";
SECRET = "02GyO3wXLrPtxYPrd5BngIbarQvR1PoFyHw74hl4CCx4sCJgWKqQv2cDhMRG/llJ+n99MuMuply/yT10HYATbg==";
BUILD_NUMBER = "e53f92ae29";
BUILD_TIME = "09.08.2017 9:22";
-They finally merge all the SWFs into the Main.swf
-Removed CityResourcesModel workaround
-Created static data for units
I still don't know why they don't create static data jsons for everything, also most of their jsons responses have useless parameters (just for easy client-side VO replacing?)

Just followed the manual and decrypted main.swf, but got a different secret? But it seems yours is working fine...

@bombcheck Hmm that is interesting, ill do it tonight and then we can have 3 data points and see who is wrong ;)
@alucardeck You can always submit a PR to change these values in [README.md] :)(https://github.com/m3talstorm/foe-decryption/blob/master/README.md)

Yeah, tried the one I got from the SWF: Did not work...

@m3talstorm ah, good to know :), but I miss a forum feature in github project
@bombcheck well, my bot is working, else my hash value would be probably wrong.
Inside the Main.swf you will find Version.as file, check if its
public static const VERSION:String = "1.107";
Yours could be from a different version

@alucardeck
Yes, your hash is working fine for me. The one I extracted by myself does not.
Used the links from the README to get the files... Then decrypted them. Where can I find Version.as?

Ah got it. It is under de/innogames/strategycity/Version.
BUILD_NUMBER:String = "5d663dc16e";
BUILD_TIME:String = "26.07.2017 10:53";
VERSION_SECRET:String = "t7vCISxcfFgoUFZkLevyhNokShuc7rBIEkcUuvlEhWC79h1eTVL5GD6jS/oweSgTghqdBLdY5lEPDZlHXWyrGQ==";
VERSION:String = "1.106";
Seems that https://foeen.innogamescdn.com/swf/Main.swf is pointing to an old version...

Quick note. I get the following for 1.107, same as OP:
02GyO3wXLrPtxYPrd5BngIbarQvR1PoFyHw74hl4CCx4sCJgWKqQv2cDhMRG/llJ+n99MuMuply/yT10HYATbg==

It works for me with the version/secret provided by OP. Also, I didn't change the timestamp (I thought it changed between versions) and it works.

The timestamp is not needed. If you read the readme for this repo, you'll see that the algorithm is:
md5(user_key + secret + encodedJson)

The version & timestamp are purely there for convenience sake - to help you identify what's "latest" without doing the operation yourself. :)

You are right, thanks Mike!

Although LordMike is correct in that the version and timestamp are not needed to generate the signature, it is still passed in the HTTP headers by the game.
See: https://github.com/m3talstorm/foe-bot/blob/master/foe/request.py#L50
The version passed in the HTTP headers determines which secret is used server side, their API supports older clients. E.g If you have a 1.106 secret and you put 1.106 as your version in the HTTP headers you can still do stuff, even if the latest version of the game is 1.107.
The timestamp is passed in the HTTP headers to try and make it look like a browser is doing the request as much as possible ;)

We use these timestamps in our projects to the trick browser caching when deploying new stuff such as JS- or CSS-files.

Oh right - forgot about those headers... Oh well :P


feat: allow setting bandits initial configuration

labels: mergeable
Fixes: #issue
Motivation and Context
Description
Allow passing initial configuration for bandits.
How has this been tested?

Thanks, great stuff! Let's bump versions and publish after this change is in.


Need Help on Understanding how this works

Could you guide me on how to remove the character name from each message? I’m unsure about the changes needed in the configuration or rules. Also, I’ve noticed that the bot generates a context conversation in the command line that doesn’t reflect our actual conversation and isn’t relayed to the Telegram bot. I’m finding this confusing.
I’m aiming for a realistic chat experience without any additional features like buttons. Could you advise me on the settings I need to adjust to achieve this?
Lastly, does the system function correctly when LoRas are loaded in?

Could you guide me on how to remove the character name from each message? I’m unsure about the changes needed in the configuration or rules.

this is hardcoded behaviour. You can try to change this in inside TelegramBotWrapper.py, but I am not sure this will work properly.

Lastly, does the system function correctly when LoRas are loaded in?

I havent tested it. This shold working in api generator mode, I think.

I’m aiming for a realistic chat experience without any additional features like buttons. Could you advise me on the settings I need to adjust to achieve this?
This looks like something you want... Chat. Buttons hidden. Conversation like with human. But "Bot:" still shown
Also, curently there is no way to switch off prefixes (++, +, -- e.t.c.)

configs.zip

upd.
Correct telegram_config file, previous is for llm_telegram_pot repo. Repos similiar, but have little differencies.
telegram_config.zip

i updated my Config to make it similar to what you sent here but it still has the button. and still for some reason once i text the a.i it generates a context.. and some of the replies are empty like on the CMD it would write, generation error: custom_token_bans.. and on telegram the bot just writes the "characters name : "
its empty ?

Change in telegram_user_rules.json all true to flase excepr last line (except message). If this wont help - idk how to help.
generation error: custom_token_bans - interaction problem between textgenerator and code. Perhaps, this using api generator type can fix this problem. p.s. seems splved #138 by clean instal


chore(main): release 2.1.0

:robot: I have created a release beep boop
2.1.0 (2024-01-08)
Features

main: manage active outputters (24d5fed)
output: add --stdout.iterations option (84a0074)

Bug Fixes

main: wait 100msec after sending STP (c123277)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/northeye/chissoku/releases/tag/v2.1.0 :sunflower:


Listen all events from the gateway

Hello)
Thank you for this great library)
I want to listen to all messages from the gateway. Unfortunately, I can see that public Observable<List<OpenSession>> listen(OpenMessage... requests) hasn't been implemented yet. Will you implement this method in near time?
If don't, could you tell me, how can I implement this opportunity (listen to all messages from the gateway)?

Hello @EugeneYarem19 , sorry for the late answer!
Correct this method is not implemented and unfortunately I do not have time to make any progress on this at the moment. So any help is more than welcome.
As far as I know it should be enough to send a Channel.EVENT frame to start receiving notifications, but honestly I never investigate it too much.
If you want to implement it I would recommend to look at the official documentation first and then try to connect manually using netcat for example and see what are the frames that you receive back and if they are reliable. Also if you can filter them somehow, to don't get overflowed.
After that you might be able to reuse some of the utility methods that are already there to parse the frames.
Let me know if you need more help and thanks again for your interest in this project!

@niqdev , I have an idea how it can be implemented. I'm going to try it on Saturday. I'll write to you on the next week)

Awesome, thanks!

So...
I've tried to listen the gateway via react-native-tcp (I'm a React Native Developer) and it work's. This question is closed for me now. Sorry, I won't be able to help you with implementing of this feature in your library in near time. But I can say that you have almost all parameters and functions to implement it) Good luck)

Thanks for your feedback!


[✨] add mousewheel control to carousel

Is your feature request related to a problem?
I'd use qwik-ui's carousel if it had this feature.
Describe the solution you'd like
like swiperjs does
https://swiperjs.com/demos#mousewheel-control
Describe alternatives you've considered
I'm sticking with swiperjs today; with a <script async> call.
Additional context
No response

Hey Patrick! Looks like I forgot to record my microphone here, but this should be out in the next release of Qwik UI. In the video is how I added the feature in case you would like to make an effort to customize the carousel to more swiper features.
Appreciate the suggestion and merry christmas 🎄
https://github.com/user-attachments/assets/e423b0d9-fcbe-442d-8c60-9e3a77c7d37a

that's amazing! thanks for the backstage footage also 🤣
happy christmas 👋🏾


Langflow saves SQLite Database in venv

Describe the bug
Langflow saves the SQLite database (langflow.db, langflow.db-shm and langflow.db-wal) of its flows directly in the venv, e.g. /app/.venv/lib/python3.12/site-packages/langflow/langflow.db*. If the venv is deleted or renewed, the data will also be deleted. It certainly makes sense to save the database outside the venv, possibly in the ".cache" directory?

@dosu: Thanks for the answer. In the .env file, however, it is sufficient to set one setting, either LANGFLOW_SAVE_DB_IN_CONFIG_DIR=true or LANGFLOW_DATABASE_URL=sqlite:///.... It is not necessary to set both settings.
Just for the sake of completeness: An sqlite URL to a file must use an absolute path. The tilde for the home directory cannot be used.


Click on Pushnotification , Application launched but pushObject.on('notification'). not triggering

Expected Behaviour
Push notification received when App is minimized and clicking on Notification, app should launch and
pushObject.on('notification').  should trigger
Actual Behaviour
Push notification received by phone when app is minimized and clicking on Push notification, its launches App, but pushObject.on('notification'). not triggering
Reproduce Scenario (including but not limited to)
Steps to Reproduce
Platform and Version (eg. Android 5.0 or iOS 9.2.1)
Android 7
(Android) What device vendor (e.g. Samsung, HTC, Sony...)
Motog4plus ( not tested other cells)
Cordova CLI version and cordova platform version
phonegap-plugin-push": "1.10.5"
"@ionic-native/push": "4.2.1",
Ionic Framework: 3.9.2
Ionic App Scripts: 3.1.0
Angular Core: 5.0.0
Angular Compiler CLI: 5.0.0
Node: 6.10.3
Payload
{
   "to":"fCMtoken",
   "notification":{
      "title":"Alert",
      "body":"Vehicle"
   },
   "data":{
      "notificationObject":{
         "Vehicle":"Vehicle xxxx serviced "
      },
      "summary":{
         "driverName":"Praveen",
         "vehicleId":108
      }
   }
}

JS
initPushNofiticationRegistrationProcess() {
    const pushObject: PushObject = this.push.init({
        android: {
            senderID: SENDER_ID,
            vibrate: true
        },
        ios: {
        },
        windows: {}
    });

    this.push.hasPermission().then(data => {
        console.log(data);
        if (data.isEnabled) {
            console.log('isEnabled');
        } else {
            this.showInfoToaster('Notification is blocked by User !!! Please Enable');
        }
    }).catch(err => {
        console.log(err);
    })

    pushObject.on('registration').subscribe((response) => {
        console.log('token', response['registrationId']);
    });

    pushObject.on('notification').subscribe((response) => {
        pushObject.getApplicationIconBadgeNumber().then(c => {
            console.log(c);
        })
        this.events.publish('notification');
    });

    pushObject.on('error').subscribe((e) => {
        console.error('push notifi-error', e['message']);
    });
}

According to you, your payload is:
{
   "to":"fCMtoken",
   "notification":{
      "title":"Alert",
      "body":"Vehicle"
   },
   "data":{
      "notificationObject":{
         "Vehicle":"Vehicle xxxx serviced "
      },
      "summary":{
         "driverName":"Praveen",
         "vehicleId":108
      }
   }
}

But according to the docs on the payload section for android:

If you send a payload with a mix of notification & data objects like this:
...
When your app is in the foreground any on('notification') handlers you have registered will be called. If your app is in the background, the notification will show up in the system tray. Clicking on the notification in the system tray will start the app and your on('notification') handler will not be called as messages that have notification payloads will not cause the plugins onMessageReceived method to be called.

So, to make your payload work on all use cases, you need to use the recommended format (from the docs), which is to not use a notification section, and instead send everything on the data section.
Make note, this only applies to Android. The docs have all the details on the differences between APNS/GCM/FCM payloads to both platforms, but we welcome suggestions to improve readability there.
I'm closing this, but if after using the recommended format the issue still applies, feel free to reopen with  more details.

Tried the below format , added user info in 'data' section as said in the doc , still the app is not responding   for  "  on('notification')  "
{
"to":"fCMtoken",
"notification":{
"title":"Alert",
"body":"Vehicle"
},
"data":{
"driverName":"Praveen",
"vehicleId":108,
"notificationObject":{
"Vehicle":"Vehicle xxxx serviced "
},
"summary":{
"driverName":"Praveen",
"vehicleId":108
}
}
}

I still see the notification section on your payload. The point was/is exactly to remove that, and only use the data section.

Tried removing the notification section and kept all user info  in 'data' key,
due to this change Push notification had failed to appear on system tray.
Without appearing on system tray how user will navigate to app ?
Do I need to fire simultaniously two different payload one for data n one for notification ??
Thanks in Advance

@praveenpds this payload should show up in the tray:
{
   "to":"fCMtoken",
   "data":{
      "title":"Alert",
      "body":"Vehicle",
      "driverName":"Praveen",
      "vehicleId":108,
      "notificationObject":{
         "Vehicle":"Vehicle xxxx serviced "
      },
      "summary":{
         "driverName":"Praveen",
         "vehicleId":108
      }
   }
}

If you are not seeing this payload show up in the tray we'll need to see the logs from adb logcat | grep Push when you send a push to the device.

Super it works fine :)


[Removal]: Round Buby Meerkat

Hotspot Name
Round Buby Meerkat
Hotspot b58 Address
11Js5PfmJSzbGLNLXaMpQS2bRBVPuhtKcWQFrNSWjzRVuBpzbN7
Discord Handle
No response
Hotspot Manufacturer
Bobcat
Removal Reason
My hotspots is a true deployment,that can be detected or mappping at any time. I request that my my hotspots be removed from the rejected list, and I hope helium can approves it.
Modifications
No
Extra forwarders
No
Extra antennas
No
Additional Information
Here are my hotspots actual photos
https://user-images.githubusercontent.com/64061024/156875835-b792212d-7401-4aed-8cf3-1e9b3c0042fe.mp4

We have taken into account this removal request in the next iteration of the denylist. It may take some time before it's processed. This is an automated comment due to the volume of addition and removal requests. A future tool is in the works to provide more insight into the analysis approach made by the Helium team and community members that maintain this list.


[@bara/core] feat: implement useStream

[x] Register using pure function
[ ] Prevent useStream of duplication name
[ ] Make sure all the stream registers has been registered before registering any trigger

Duplicate stream

Duplicate stream

This has been fixed in #18


Reconcile error for HetznerBareMetalHost and more than one network interface

/kind bug
What steps did you take and what happened:
I'm trying to set up k8s cluster with only one node using hetzner-baremetal-control-planes flavor. After generating cluster and adding HetznerBareMetalHost I don't see any detail about host hardware when doing kubectl describe hetznerbaremetalhost.
Here is the log from caph-controller-manager:

Log
  {
  "level": "ERROR",
  "time": "2023-09-21T11:18:53.496Z",
  "file": "controller/controller.go:324",
  "message": "Reconciler error",
  "controller": "hetznerbaremetalhost",
  "controllerGroup": "infrastructure.cluster.x-k8s.io",
  "controllerKind": "HetznerBareMetalHost",
  "HetznerBareMetalHost": {
    "name": "de1459",
    "namespace": "de-dev"
  },
  "namespace": "de-dev",
  "name": "de1459",
  "reconcileID": "9283fd2c-9da9-4274-aaae-ffbea85dbf64",
  "error": "failed to reconcile HetznerBareMetalHost de-dev/de1459: action \"registering\" failed: failed to get hardware details: failed to obtain hardware details Nics: failed to unmarshal {\"name\":\"eth0\",\"model\":\"Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)}. Original ssh output name=\"eth0\" model=\"Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)\nIntel Corporation I350 Gigabit Network Connection (rev 01)\" mac=\"f0:2f:74:94:a2:41\" ip=\"<IP_ADDRESS>/26\" speedMbps=\"1000\"\nname=\"eth0\" model=\"Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)\nIntel Corporation I350 Gigabit Network Connection (rev 01)\" mac=\"f0:2f:74:94:a2:41\" ip=\"2a01:4f8:262:265f::2/64\" speedMbps=\"1000\": unexpected end of JSON input",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:226"
}

What did you expect to happen:
Reconcilation completed successfully
Anything else you would like to add:
I assume that's due to this line: https://github.com/syself/cluster-api-provider-hetzner/blob/v1.0.0-beta.22/pkg/services/baremetal/client/ssh/ssh_client.go#L144
When executed on host, it returns 2 lines:
root@rescue ~ # lspci | grep net | awk '{$1=$2=$3=""; print $0}' | sed "s/^[ \t]*//"
Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Intel Corporation I350 Gigabit Network Connection (rev 01)

and the script output is like that:
root@rescue ~ # bash nic-info.sh
name="eth0" model="Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Intel Corporation I350 Gigabit Network Connection (rev 01)" mac="f0:2f:74:94:a2:41" ip="<IP_ADDRESS>/26" speedMbps="1000"
name="eth0" model="Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Intel Corporation I350 Gigabit Network Connection (rev 01)" mac="f0:2f:74:94:a2:41" ip="2a01:4f8:262:265f::2/64" speedMbps="1000"

Environment:

cluster-api-provider-hetzner version: v1.0.0-beta.22
Kubernetes version: 1.25.12
OS (e.g. from /etc/os-release):  debian 12

As an idea, something like that could be used instead:
lspci -s $(ethtool -i $iname | grep bus-info | awk '{print $2}') | cut -d ':' -f 3

@guettli please have a look here

I ran into a similar error on a RX220 Host with the following versions:

cluster-api-provider-hetzner version: v1.0.0-beta.25
Kubernetes version: 1.28.2
OS (e.g. from /etc/os-release): debian 12

Log:
{"level":"ERROR","time":"2023-10-06T17:33:42.789Z","file":"controller/controller.go:324","message":"Reconciler error","controller":"hetznerbaremetalhost","controllerGroup":"infrastructure.cluster.x-k8s.io","controllerKind":"HetznerBareMetalHost","HetznerBareMetalHost":{"name":"bm-arm-01","namespace":"default"},"namespace":"default","name":"bm-arm-01","reconcileID":"3f8d7595-bc7f-4174-97d0-b7b49efbc96d","error":"failed to reconcile HetznerBareMetalHost default/bm-arm-01: action \"registering\" failed: failed to get hardware details: failed to obtain hardware details Nics: failed to unmarshal {\"name\":\"eth0\",\"model\":\"Intel Corporation I350 Gigabit Network Connection (rev 01)}. Original ssh output name=\"eth0\" model=\"Intel Corporation I350 Gigabit Network Connection (rev 01)\nIntel Corporation I350 Gigabit Network Connection (rev 01)\" mac=\"88:88:88:88:88:88\" ip=\"<IP_ADDRESS>/26\" speedMbps=\"1000\"\nname=\"eth0\" model=\"Intel Corporation I350 Gigabit Network Connection (rev 01)\nIntel Corporation I350 Gigabit Network Connection (rev 01)\" mac=\"88:88:88:88:88:88\" ip=\"2a01:2a01:2a01:2a01::2/64\" speedMbps=\"1000\": unexpected end of JSON input","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:226"}

Above log line, pretty printed:
❯ xclip -o | yq -P
level: ERROR
time: "2023-10-06T17:33:42.789Z"
file: controller/controller.go:324
message: Reconciler error
controller: hetznerbaremetalhost
controllerGroup: infrastructure.cluster.x-k8s.io
controllerKind: HetznerBareMetalHost
HetznerBareMetalHost:
  name: bm-arm-01
  namespace: default
namespace: default
name: bm-arm-01
reconcileID: 3f8d7595-bc7f-4174-97d0-b7b49efbc96d
error: |-
  failed to reconcile HetznerBareMetalHost default/bm-arm-01: action "registering" failed: failed to get hardware details: failed to obtain hardware details Nics: failed to unmarshal {"name":"eth0","model":"Intel Corporation I350 Gigabit Network Connection (rev 01)}. Original ssh output name="eth0" model="Intel Corporation I350 Gigabit Network Connection (rev 01)
  Intel Corporation I350 Gigabit Network Connection (rev 01)" mac="88:88:88:88:88:88" ip="<IP_ADDRESS>/26" speedMbps="1000"
  name="eth0" model="Intel Corporation I350 Gigabit Network Connection (rev 01)
  Intel Corporation I350 Gigabit Network Connection (rev 01)" mac="88:88:88:88:88:88" ip="2a01:2a01:2a01:2a01::2/64" speedMbps="1000": unexpected end of JSON input
stacktrace: |-
  sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:324
  sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:265
  sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /src/cluster-api-provider-hetzner/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:226

Is there any work in progress regarding this issue? If not, would you accept a pull request?

@benedikt-bartscher yes, a PR is welcome. BTW, do you have an idea how to reproduce this error? Is there a way to create a second (fake) network interface somehow? Then I could validate you PR manually.

Hey @guettli thanks for your response. You could alias lspci to echo some test data. I am not aware of any other trick which results in a "fake" NIC appearing in lspci/ethtool. Aren't your e2e tests sponsored by Hetzner? Maybe they can provide you a server with 2 NICs. If not, I could provide you with one of our machines for some coding/testing for free.

They will, no problem. If you can open a PR, we will be able to test it as well!

@Lenikas is it possible to schedule a call for further debugging?
@guettli please have a look into this in the upcoming week.

@Lenikas can you please post the output of these commands:
ip a

ethtool "*"

lspci

thank you!

@Lenikas can you please post the output of these commands:
ip a

ethtool "*"

lspci

thank you!

Hello @guettli, thank you for replying!
This is output from server EX130-R type:
ip a:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 9c:6b:00:45:9c:5e brd ff:ff:ff:ff:ff:ff
    altname eno1
    altname enp5s0f0
    inet <IP_ADDRESS>/26 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2a01:4f9:3070:1e05::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::9e6b:ff:fe45:9c5e/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 9c:6b:00:45:9c:5f brd ff:ff:ff:ff:ff:ff
    altname eno2
    altname enp5s0f1
4: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether ce:b3:36:d9:a6:a2 brd ff:ff:ff:ff:ff:ff

ethtool: "*"
Settings for eth0:
	Supported ports: [ TP ]
	Supported link modes:   100baseT/Full
	                        1000baseT/Full
	                        10000baseT/Full
	                        2500baseT/Full
	                        5000baseT/Full
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  100baseT/Full
	                        1000baseT/Full
	                        10000baseT/Full
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: 1000Mb/s
	Duplex: Full
	Auto-negotiation: on
Settings for eth1:
	Supported ports: [ TP ]
	Supported link modes:   100baseT/Full
	                        1000baseT/Full
	                        10000baseT/Full
	                        2500baseT/Full
	                        5000baseT/Full
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  100baseT/Full
	                        1000baseT/Full
	                        10000baseT/Full
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: Unknown!
	Duplex: Unknown! (255)
	Auto-negotiation: on
Settings for usb0:
	Supported ports: [  ]
	Supported link modes:   Not reported
	Supported pause frame use: No
	Supports auto-negotiation: No
	Supported FEC modes: Not reported
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Advertised FEC modes: Not reported
	Speed: Unknown!

Hello @batistein, @guettli!
If relevant, we can schedule a meeting. Alternatively, we can suggest transitioning our communication to a different platform if it's more convenient for you. Additionally, we can grant you SSH access to the server for debugging purposes.
How long do you think it might take to resolve the issue? It's important for our team to understand this to plan our next steps. Unfortunately, our team lacks sufficient expertise in Go to quickly resolve this issue.
If you need any further information, we're ready to provide it.
Thank you!

@Lenikas we created a draft which should make the error go away.
Do you need the NIC data which gets gathered by the script? Because at the moment the script nic-info.sh does not work reliably. But I guess you don't need these values, and you just want the provisioning to succeed.

@guettli
Yes, at the moment, we simply need a fix to ensure that provisioning completes successfully.
However, we are unsure where this information may be needed in the future. Perhaps you have some ideas or is it related to some functionality of the cluster-api-provider-hetzner?
Thank you for the responsive communication!

@Lenikas the PR is merged, you can test the new caph image by updating the caph deployment in your management cluster.
Image:  ghcr.io/syself/caph-staging:sha-c6fd5bb
See: https://github.com/syself/cluster-api-provider-hetzner/pkgs/container/caph-staging/190282019?tag=sha-c6fd5bb
Please tell us if this works for you. Thank you.

@guettli Hello
I apologize for the delayed response.
Yes, I have checked the built image, it works. The provisioning completes successfully, and the nodes are added to the cluster.
Thank you so much!


Translation server and demo frontend

The existing translation server from OpenNMT-py was refurbished.
A demo frontend was implemented using streamlit.

The backports.zoneinfo dependency should be conditioned on python<3.9

Embarks partial fix for #7

@Waino do we merge?


Adding kind for node[] fields where possible

For at least some of the node[] we could give a kind and so that array's elements would have a proper type in documentation, Java code, etc.
Some example:

MatchWriteNode#targets, (I think) always LocalVariableTargetNode

Some counter example:

ParametersNode#requireds, not always RequiredParameterNode, can also be MultiTargetNode

In some cases it's not possible because multiple classes are possible, or a MissingNode is possible.

Yes I agree. I think we should also use this opportunity to expand the syntax to support a set of nodes for languages where a union makes sense.

Indeed, a union would be super helpful, even if it only ends up in documentation.
(Java doesn't have union types unfortunately)

Probably we can consider this done when https://github.com/ruby/prism/pull/2101#discussion_r1445812790 is merged.


Explain in group email templates when a membership has expired.

@johngian thoughts? should we include this in our templates?

@johngian thoughts on this?


POST with files and timeout does both?

Hi, im making a requests.post call with sometimes large files object and a 5s timeout. Im observing that even if the call does indeed raise a timeout exception, the full file still ends up being sent.. Somehow.
Is this intentional or are there any nuances to using timeout with files?

A timeout is a complex thing. In this case, the timeout is applied per socket operation. That means each send, regardless of how much data, and each read.
In this case you are likely timing out on the read. That will lead to the behaviour you’re describing.
If you want a total timeout you can arrange it, but it’s definitely tricky.

Hi, so just to be clear, if uploading the file actually takes minutes, but the request is constantly sending sone data, would the timeout trigger, or does it see that data is being sent, so all is good?

So long as at least one byte reaches the wire per timeout interval, it would think all is good.


user_topics: Rename muted_topics.js to user_topics.js.

This renames muted_topics.js to user_topics.js as we'll be switching over to using user_topics instead of muted_topics.

Merged, thanks @akashaviator!


Bad Bug in genome browser

I am doing an analysis where I need to look at contig 162 but it only allows me to select from the first 100 contigs.
Also it still bothers me that the original contig names are not retained.  I have no assurance that my contig 162 is the patric contig 162.

We discussed this yesterday. Increasing the number of contigs displayed in the dropdown will not help as there are genomes with more than 1000 contigs.
@aswarren, We decided that we will provide links to genome browser from the sequence table. So, if you like to see a specific contig in genome browser, you can go to sequences tab, search for you contig of interest, and then jump top genome browser from there.
I will see if we can change the data processing pipleine to retain original contig ids as labels or descriptions, so that they can be displayed in the sequence table and made searchable.
-Maulik

The immediate bug with the drop down limit has been fixed. The suggested implementation requires further discussion.

@hyoo In the sequences tab, when a single sequence is selected, provide link to the genome browser...similar to the one you provided from the contig blast.

This really should be a Sequence View with Genome Browser tab within.

On Nov 30, 2016, at 2:03 AM, Maulik Shukla<EMAIL_ADDRESS>wrote:
@hyoo https://github.com/hyoo In the sequences tab, when a single sequence is selected, provide link to the genome browser...similar to the one you provided from the contig blast.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/PATRIC3/patric3_website/issues/1211#issuecomment-263799082, or mute the thread https://github.com/notifications/unsubscribe-auth/AALoblVxZ5Ja_dO9XTl7H2b7UXY5Ovo4ks5rDR-5gaJpZM4KnVJ0.

Implemented in https://github.com/PATRIC3/p3_web/pull/310
@mshukla1 Thats not really what I meant by discussion.


CorsService doesn't handle null for origin

Sometimes in some situations null is passed as the origin -- handle that better.

fixed


Library sepearation

Working on #25.
Library separated to following subPackages:

core: core of the DCV, computer vision algorithms;
io: image and video I/O;
plot: visualization package.

Main motivation:

have modular library setup, and by that less dependencies for core module;
having core module not depend on C libraries.
... there were more, can't remember at the moment..

SubPackage dependencies:

core

mir

io

core
imageformats
ffmpeg-d

plot

core
ggplotd (optional)

Configurations:
To make ggplotd optional, ggplotd configuration has been added to dcv:plot package. And to make it optional on the global level, ggplotd configuration has been added to root package. So to use ggplotd stuff, subConfiguration has to be defined, like in some of the examples.

Pinging @DmitryOlshansky @9il @henrygouk all of you, since I feel this is a big change, and I'd like all of you to review it, and tell me what you think.
Note: ggplotd configuration's name is not so lucky in mu opinion, but I'm not sure how better to name it. If anyone has a suggestion, fire away.
One more note: There seam to be a bug in Dub in making this kind of setup. When setting up sub-packages like this, unittest-cov build gets messed up. This is actually copied from vibe.d's dub config as example, and there they offer a workaround. This is also applied here, and seems to be working like a charm (until proven otherwise...).

Closes #25

Current coverage is 97.04% (diff: 100%)

Merging #48 into master will increase coverage by 23.71%

@@             master        #48   diff @@
==========================================
  Files            25          1     -24
  Lines          2613        169   -2444
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           1916        164   -1752
+ Misses          697          5    -692
  Partials          0          0

Powered by Codecov. Last update 1d5e427...ec24f84

So, coverage is off. I'm trying to fix it, if anyone knows the straight solution, please help.

I think adding configuration is overengineering here. It is faster for user to add both sub-packages, then to define configuration.
Please split video (requires C) and image  (not requires C) io.

I think adding configuration is overengineering here. It is faster for user to add both sub-packages, then to define configuration.

To which configuration do you refer - one in the root? If so, than user which plans to use whole DCV, has to define either all three packages separately, either the root one and the plot, latter to define the subConfiguration, like so:
"dependencies": {
    "dcv": "",
    "dcv:plot": ""
},
"subConfigurations":{
    "dcv:plot": "ggplotd"
}

Of course he can define separated all three - maybe you're right, that'd be the cleanest way. @henrygouk and @DmitryOlshansky please tell us what you think of this matter.

Please split video (requires C) and image (not requires C) io.

I like this. I will do it, @henrygouk @DmitryOlshansky please tell us if you have anything against it.

Of course he can define separated all three - maybe you're right, that'd be the cleanest way. @henrygouk and @DmitryOlshansky please tell us what you think of this matter.

oh, never mind. I misunderstood config

Please split video (requires C) and image (not requires C) io.

How do we name image I/O and video I/O packages? Having only image and video might be misleading. Maybe ioimage and iovideo, but that seems too much. :) Ideas?
Are they subPackages of the package io, or do we simply split it into two packages? It'd be maybe better to make them subPackages of io since like that we'd follow existing project structure - that way it would be dcv:io:image and dcv:io:video, which seems ok to me. Otherwise I suggest we change that structure to fit the configuration.

another option is io-image, io-video

Is the image io portion of the library likely to remain free of C dependencies? Currently imageformats only supports writing 8 bit images, and the easiest way to add support to DCV for higher bit depth encoding would be to simply wrap an existing C library. Conveniently, ffmpeg is able to do this.

Is the image io portion of the library likely to remain free of C dependencies

My original plan was to do exactly what you propose- write minimal bindings to well known C libraries and by that support whole lot of formats with industry proven backend. Imageformats does support 16bit png and tga I think, but nevertheless I feel wrapping those libraries would be better choice. But that's a discussion for another thread, which I'll start latter in the day.
Maybe postpone io splitting because of this dilemma?

OK

Have user ability to his own IO library?

Have user ability to his own IO library?

That would be really nice but I'm not sure how to realize this the right way. I suppose you think of some middle layer between dcv and io lib? Alternatively user can always handle himself the io,  slice read data and work along in dcv.

Alternatively user can always handle himself the io, slice read data and work along in dcv.

Yes, this is what i whant.

Imageformats does support 16bit png and tga I think

Imageformats can read 16 bit, but not write it.

Maybe postpone io splitting because of this dilemma?

If by this you mean keeping the video and image packages as a single subpackage of DCV, then I agree.

Ok I'll make an issue about this latter in the day then we'll discuss the details.
I still have fix the codecov, in the meantime...

Coverage is fine, so I'd like to merge this one. We haven't heard from @DmitryOlshansky, but I suppose he'd be fine we these changes.
Thanks guys!


feat: remove banks, hyperstructures and leaderboard

blurred the tabs on the Eternum map with "Coming Soon"
commented out travel to hyperstructure
commented out conquered hyperstructure

resolves #406