calc_LOS_signal2 : choice of strategy and mutualization of sampling

In _GG:

LOS_calc_signal() should be replaced by LOS_calc_signal2()

LOS_calc_signal2() has been drafted (branch from Issue61_Imas2TofuMoreRobust)

It introduces: a choice between three strategies (minimize user's function calls by using vectorized routines for small number of LOS, minimize memory usage by looping for large number of LOS)

It needs both the existing LOS_get_sample (for minimize = calls) and a single-LOS version of sampling (for minimize = memory)

Making that single-LOS version of LOS_get_sample() may require revizing a bit the _sampling_tools.pxd to introduce inlined single-LOS unparallelized versions of all sampling methods.

This update will make computation of synthetic signal faster / more efficient (especially for large number of LOS) and more robust vs user's functions (if ill-defined, we can minimize the number of calls).
It will also make synthetic signal computation from Plasma2D objects easier and more natural

@lasofivec , now branch Issue61_Imas2TofuMoreRobust is up-to-date with devel (thanks to PR #114 )
You can have a look at the drafted solution in _GG02.pyx (LOS_calc_signal2) from this branch

Branch Issue108... is failing because of some changes in Issue61_Imas2TofuMoreRobust with python2
See this travis fail report
INFO conda_build.build:bundle_conda(891): Packaging tofu-<IP_ADDRESS>1.g392e0c0-py27h39e3cac_0
  File "lib/python2.7/site-packages/tofu-1.4.0_319_g7fbfab5-py2.7-linux-x86_64.egg/tofu/data/_core.py", line 1889
    data = chronos[:,modes] @ (s[modes,None] * topos[modes,:])
                            ^
SyntaxError: invalid syntax

On Python3 there are other fails that come from the original branch, the only one that comes from LOS_calc_signal is the following.
======================================================================
FAIL: tofu.tests.tests01_geom.tests03_core.Test03_Rays.test10_calc_signal
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/tests/tests01_geom/tests03_core.py", line 870, in test10_calc_signal
    fs=(12,6), connect=connect)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/geom/_core.py", line 4590, in calc_signal
    connect=connect)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/geom/_core.py", line 4510, in _calc_signal_postformat
    osig = tfd.DataCam2D(**kwdargs)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/data/_core.py", line 157, in __init__
    super(DataAbstract,self).__init__(**kwdargs)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/utils.py", line 973, in __init__
    self._init(**kwdargs)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/data/_core.py", line 610, in _init
    self._set_ddataRef(**kwddataRef)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/data/_core.py", line 631, in _set_ddataRef
    lout = self._checkformat_inputs_ddataRef(**kwdargs)
  File "/home/travis/miniconda/lib/python3.6/site-packages/tofu/data/_core.py", line 274, in _checkformat_inputs_ddataRef
    assert ndim in [2,3]
AssertionError

However, this issue come from the method _calc_signal_postformat where we are trying to build a camera to 2D from the integrated signal, however the signal is 1D. I rather not change this myself and pass it on to you. I will make a pull request and close this issue.


No IP address is assigned with Ubuntu example?

System Information
Not sure if it's related to https://github.com/dmacvicar/terraform-provider-libvirt/issues/312
Full log of TF_LOG=debug terraform apply here: https://gist.github.com/chrissound/d30e2a6a7863d53deeacc63a6a588fe5
$ terraform refresh
libvirt_volume.ubuntu-qcow3: Refreshing state... (ID: /var/lib/libvirt/images/ubuntu-qcow3)
data.template_file.user_data: Refreshing state...
data.template_file.network_config: Refreshing state...
libvirt_cloudinit_disk.commoninit2: Refreshing state... (ID: /var/lib/libvirt/images/commoninit.iso;5d42cf95-cde7-b95b-e493-6b040c7c2d5b)
libvirt_domain.domain-ubuntu: Refreshing state... (ID: 76a842bf-2396-4b98-8270-964aa6ddb953)
$ terraform show
data.template_file.network_config:
  id = df16c3f9c2a6d8255670be81539ec54a42d2d59eda005c29c22efb1dd9e28909
  rendered = network:
  version: 2
  config:
  - type: physical
    name: ens3
    subnets:
      - type: dhcp

  template = network:
  version: 2
  config:
  - type: physical
    name: ens3
    subnets:
      - type: dhcp

data.template_file.user_data:
  id = 3a35a94c06c5e51de4515a34202bafc7d600081e185a589d855ca95abab5d175
  rendered = #cloud-config
# vim: syntax=yaml
#
# ***********************
# 	---- for more examples look at: ------
# ---> https://cloudinit.readthedocs.io/en/latest/topics/examples.html
# ******************************
#
ssh_pwauth: True
chpasswd:
  list: |
     root:1234
  expire: False

  template = #cloud-config
# vim: syntax=yaml
#
# ***********************
# 	---- for more examples look at: ------
# ---> https://cloudinit.readthedocs.io/en/latest/topics/examples.html
# ******************************
#
ssh_pwauth: True
chpasswd:
  list: |
     root:1234
  expire: False

libvirt_cloudinit_disk.commoninit2:
  id = /var/lib/libvirt/images/commoninit.iso;5d42cf95-cde7-b95b-e493-6b040c7c2d5b
  meta_data =
  name = commoninit.iso
  network_config = network:
  version: 2
  config:
  - type: physical
    name: ens3
    subnets:
      - type: dhcp

  pool = default
  user_data = #cloud-config
# vim: syntax=yaml
#
# ***********************
# 	---- for more examples look at: ------
# ---> https://cloudinit.readthedocs.io/en/latest/topics/examples.html
# ******************************
#
ssh_pwauth: True
chpasswd:
  list: |
     root:1234
  expire: False

libvirt_domain.domain-ubuntu:
  id = 76a842bf-2396-4b98-8270-964aa6ddb953
  arch = x86_64
  autostart = false
  cloudinit = /var/lib/libvirt/images/commoninit.iso;5d42cf95-cde7-b95b-e493-6b040c7c2d5b
  cmdline.# = 0
  console.# = 2
  console.0.source_path =
  console.0.target_port = 0
  console.0.target_type = serial
  console.0.type = pty
  console.1.source_path =
  console.1.target_port = 1
  console.1.target_type = virtio
  console.1.type = pty
  disk.# = 1
  disk.0.file =
  disk.0.scsi = false
  disk.0.url =
  disk.0.volume_id = /var/lib/libvirt/images/ubuntu-qcow3
  disk.0.wwn =
  emulator = /run/libvirt/nix-emulators/qemu-system-x86_64
  firmware =
  graphics.# = 1
  graphics.0.autoport = true
  graphics.0.listen_type = address
  graphics.0.type = spice
  initrd =
  kernel =
  machine = pc
  memory = 800
  name = ubuntu-terraform
  network_interface.# = 1
  network_interface.0.addresses.# = 0
  network_interface.0.bridge =
  network_interface.0.hostname =
  network_interface.0.mac = EE:05:09:A1:19:C3
  network_interface.0.macvtap =
  network_interface.0.network_id = e2f1a2c0-268e-47bc-bbfd-5558cdc6cd2d
  network_interface.0.network_name = default
  network_interface.0.passthrough =
  network_interface.0.vepa =
  network_interface.0.wait_for_lease = false
  nvram.# = 0
  qemu_agent = false
  running = true
  vcpu = 1
libvirt_volume.ubuntu-qcow3:
  id = /var/lib/libvirt/images/ubuntu-qcow3
  format = qcow2
  name = ubuntu-qcow3
  pool = default
  size =<PHONE_NUMBER>
  source = /home/chris/NewProjects/tfTest3Mysql/ubuntu-16.04-server-cloudimg-amd64-disk1.img

Linux distribution
Nixos 19.03
Terraform version
terraform --version
Terraform v0.11.14

Provider and libvirt versions
terraform-provider-libvirt -version
zsh: command not found: terraform-provider-libvirt

Checklist
Description of Issue/Question
Setup
(Please provide the full main.tf file for reproducing the issue (Be sure to remove sensitive information)
ubuntu.tf (I got this from the examples directory - possibly it's not the latest):
provider "libvirt" {
  uri = "qemu:///system"
}

resource "libvirt_volume" "ubuntu-qcow3" {
  name   = "ubuntu-qcow3"
  pool   = "default"
  source = "/home/chris/NewProjects/tfTest3Mysql/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
  format = "qcow2"
}

data "template_file" "user_data" {
  template = "${file("${path.module}/cloud_init.cfg")}"
}

data "template_file" "network_config" {
  template = "${file("${path.module}/network_config.cfg")}"
}

resource "libvirt_cloudinit_disk" "commoninit2" {
  name           = "commoninit.iso"
  user_data      = "${data.template_file.user_data.rendered}"
  network_config = "${data.template_file.network_config.rendered}"
}

resource "libvirt_domain" "domain-ubuntu" {
  name   = "ubuntu-terraform"
  memory = "800"
  vcpu   = 1

  cloudinit = "${libvirt_cloudinit_disk.commoninit2.id}"

  network_interface {
    network_name = "default"
  }

  # IMPORTANT: this is a known bug on cloud images, since they expect a console
  # we need to pass it
  # https://bugs.launchpad.net/cloud-images/+bug/1573095
  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }

  console {
    type        = "pty"
    target_type = "virtio"
    target_port = "1"
  }

  disk {
    volume_id = "${libvirt_volume.ubuntu-qcow3.id}"
  }

  graphics {
    type        = "spice"
    listen_type = "address"
    autoport    = true
  }
}

# IPs: use wait_for_lease true or after creation use terraform refresh and terraform show for the ips of domain

Hi @chrissound thx for issue. At first glance I don't get the issue but I'm checking the issue with mobile so we need to check more ln detail

Sure thanks. Basically I'm expecting network_interface.0.addresses.# = 0 to have an IP address listed there instead of 0.

@chrissound  do you have a network iface called default? afaik back in the time this worked @zeenix CC  as he updated last time. ( I also remember that I used multiple times that examples without any pb)
Also try after the terraform apply, terraform refresh and terraform show,
The IP behaviour is explained here https://github.com/dmacvicar/terraform-provider-libvirt/blob/19c23424ccfdecf6dfcbdd87783bf835ba5f585e/website/docs/r/domain.html.markdown#L344
Nomally we don't wait for IP , so I think it is normall that you don't have it. You need either to do terraform refresh, and terraform show

Will have to investigate this again, thanks. But just thought I'd quickly reply and say I did run the refresh and show commands after the apply (it's included in the post - first code snippet).

@MalloZup I don't think i touched anything that will affect the networking bits.

@chrissound  ok I will retry on my side to check. thx for feedback both of you !

@chrissound @MalloZup
I tried this terraform provider yesterday and had the same issue. After some digging, the below config works for me in both Ubuntu 16.04 and 18.04.
Working network_config.cfg
version: 2
ethernets:
ens3:
dhcp4: true
Description of changes
In detail, two changes are required.

Drop the initial "network:" in the file
Update the syntax to that found in the cloudinit documentation here
https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html#network-config-v2
(I couldn't pinpoint the exact date the format changed)

I wasn't able to create a new branch/ set-up a pull request.
The below require updating to be in line with newer Ubuntu images.
examples/v0.11/ubuntu/network_config.cfg
examples/v0.12/ubuntu/network_config.cfg
The Suse / Leap15 examples are also likely to be affected.

Attaching file as indentation was lost in post.
network_config.cfg.txt

@chrissound
Can you please confirm if this fixes the underlying issues with retrieving the ip per your original post.
I didn't try your exact configuration, but the above fixed my issue of not seeing the ens3 interface when logged into the vm.

thx for PR. If it fix issue we can merge. :rose:

Yup that fixes it. Many thanks to all involved!


Thin grey axes on map?

On the map, at any zoom level, there are thin grey lines like coordinate axes:

This shouldn't be present!

Woah, strange. I haven't seen this in any browsers. What browser / operating system are you using?

Firefox on a Mac... I'm not sure it always shows up, though. This isn't that important a bug!

On Feb 14, 2017, at 5:48 AM, Will Newman<EMAIL_ADDRESS>wrote:
Woah, strange. I haven't seen this in any browsers. What browser / operating system are you using?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

marking this as minor bug

Not showing up for me either!


scrollback: first character of line always bold

Which operating system does the issue occur on?
manjaro 4.14.34-1-MANJARO
If on linux, are you using X11 or Wayland?
Linux using x11
xorg-server 1.19.6+13+gd0d1a694f-2
This seems to be a bug of the scrollback branch. I can reproduce on 0d568180761d7da56e756762d5f4b1555eea6a7f
The first character of a new line seems to always be printed as bold.

EDIT: for the record: I could not reproduce this on the master branch

It seems like it's not actually bold but some other rendering issue:

Maybe what happens is that the first character is displayed twice on the same place so that transparent pixels gain more saturation...

I'm closing this since the PR has been merged.

Thanks! Unfortunately github can't close issues when the PRs haven't been merged into master yet.


Create DESIGN.md

Preview 📜

Closes https://github.com/fission-codes/fission-server/issues/151

The Single Source of Truth:tm: for the Fission Server technical design, terms, concepts, and APIs.

Flipping to RFR, but I expect that there will be questions or things that need more clarity. Have at it!

The a server account should support multiple Identifier DIDs (Root DIDs) ie. if a user doesnt want to trust google sync he may need to add multiple passkey identifiers to the same account.
Or we need to rely on device linking only.

The a server account should support multiple Identifier DIDs (Root DIDs)

@hugomrdias How are you thinking about that? We can pass a list to the create function, but all of those need to be alterted to the existence of those UCANs later. There's a section in here for what is essentially a UCAN outbox, but the other devices need to know to pull. If they're online we could do this via AWAKE pubsub, but that's not the general case.
Starting with linking and ucan:* is probably the fastest way to get up and running, and we do the "multiple parents" version later.
Thoughts?

@hugomrdias How are you thinking about that? We can pass a list to the create function, but all of those need to be alterted to the existence of those UCANs later.

im just thinking in a PUT /v0/api/account/identifier that adds an new identifier that is used to validate further GET v0/api/account

There's a section in here for what is essentially a UCAN outbox, but the other devices need to know to pull. If they're online we could do this via AWAKE pubsub, but that's not the general case.

didnt get this part

Starting with linking and ucan:* is probably the fastest way to get up and running, and we do the "multiple parents" version later.

yes!

im just thinking in a PUT /v0/api/account/identifier that adds an new identifier that is used to validate further GET v0/api/account

Sorry, I'm still not sure that I follow 😅 You mean an endpoint where people can register their DIDs with the server? How does that enable GET /v0/api/account requests?

didnt get this part

Ah, sorry. Basically if we more closely follow a capabilities model than we have historically — previously we let users tell us a DID rather than giving the agent generate it's own unique DID — we can later do things like give the agnet multiple parents or directly delegate to multiple users. That either requires that they be online, or we need to put them somewhere that they can find them later. An outbox is one such mechanism: "periodically check here for more UCANs". This store could even be unauthenticated — UCANs are technically secure in public — but getting an invication to that endpoint can act as a gossip-capable store-and-forward outbox for them to pull from and see if they have any UCANs waiting for them.

Sorry, I'm still not sure that I follow 😅 You mean an endpoint where people can register their DIDs with the server? How does that enable GET /v0/api/account requests?

User 1 registers Account 1 with Identifier 1 and now can invoke account caps, after User 1 wants to associate another Identifier 2 to Account 1 so he can invoke account caps with Identifier 2 also.
Does this make it clearer ?

we can later do things like give the agnet multiple parents or directly delegate to multiple users.

Give agents multiple parents? What do you mean, im my view agent are ephemeral and don't matter much in terms of delegation audience. That why we target Identifiers and not agents.

@hugomrdias IIUC, (and my hope is this helps clarify this), it's decent UX to allow associating multiple passkeys with a single account, correct? So, each passkey would be an 'Identifier', and supporting adding multiple is so that the UX is better for passkeys, right?

Please don't mind if I do! :eyes:

Thanks for merging @matheus23 👍

That why we target Identifiers and not agents.

We may be using these terms differently. I tend to use "agent" to mean anything with a DID, by analogy to actors (but tehse don't fit the exact definition of an actor). I don't use the term Identifier because it's so broad.

User 1 registers Account 1 with Identifier 1 and now can invoke account caps, after User 1 wants to associate another Identifier 2 to Account 1 so he can invoke account caps with Identifier 2 also.

Gotcha. I think we agree on the goal. This topic came up on a call with Philipp earlier, and we can still achieve what you're describing with the design I'm proposing. Since the delegation goes through the server, we can add and remove delegations at will without any additional mechanisms.


SOLR-16776: Disable remote streaming via system property

Details in JIRA.

I don't have time to pursue this issue any further. Thanks for the feedback, @janhoy .

Superceded by #1615


ES2015ify and require Node.js 4

I had considered using spread parameters but they are not available in Node.js 4.
The goal is to get boxen to stop installing two versions of string-width.  boxen already requires node >= 4 so updating to widest-line 2.x will be a non-breaking change.  I can submit a PR to boxen if you like.
If you want me to update the package.json version I can do that, but looking at your other commit logs it looks like you always do that in a separate commit.

Thanks :)


Add operations for manipulating meta data

This PR adds an operations to manipulate Labels as well as the Labels columns stored in TensorMaps as keys, samples or properties.

:books: Documentation preview :books:: https://equistore--299.org.readthedocs.build/en/299/

Sorry for the many force pushes. The three functions should be there now.

I renamed the functions and introduced the Labels methods that I needed.
Let me know what you think.

Thanks for the review @jwa7 ! I put the length check in the beginning of the function. We don't have to do this for every block.
I will now do the port of the Labels method for equsitore-torch.

I did the renaming. If you two @Luthaf and @jwa7 are happy we can merge this.

@Luthaf if you are also happy you can merge this.

Looks good!


content appears incomplete on streaming large files from zip

In
https://discourse.gbif.org/t/type-specimen-castype1652-found-via-filtered-query-https-doi-org-10-15468-dl-xf6ahb-but-not-in-open-access-gbif-data-product-https-doi-org-10-15468-dl-pk3trq/3884/2
@mattblissett noted that

[...]
From your script, you might be streaming the decompression of the Zip file. I think libarchive has a bug when streaming-decompressing large Zip archives produced by streaming compression in Java, although it loses only the last few kB of data, not the many megabytes which a total row count of 2.07 billion would suggest. Could something like that have happened?
[...]

thanks again @mattblissett for taking a stab at reproducing my observations.
Hoping to get to the bottom of this sooner or later.

In processing the retrieved zip with hash://sha256/c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97 using unzip tools, I was able to count 2.30 billion records
unzip -p c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97 0015281-230224095556074.csv\
 | pv -l\
 > /dev/null

yielded
2.30G 1:43:31 [ 370k/s]

This alternate method to count lines in a known resource (e.g., content id is known) suggests that the alternate method used in https://gist.github.com/jhpoelen/569a3a787f6da542c8202ecddbacf580 or
preston cat 'zip:hash://sha256/c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97!/0015281-230224095556074.csv'\
 | pv -l

which initially yielded
2.07G 5:16:05 [ 109k/s]

in the workflow described by https://gist.github.com/jhpoelen/569a3a787f6da542c8202ecddbacf580

@MattBlissett thanks for confirming that our content identifiers match. This helps me in narrowing down the sources of the line count discrepancies observed earlier.

Was able to confirm that preston is likely not the source of the miscount. . .
$ preston cat 'zip:hash://sha256/c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97!/0015281-230224095556074.csv' | pv -l > /dev/null
2.30G 1:03:25 [ 605k/s] [      <=>                                             ]

so this leaves the idea that mlr --tsvlite filter '$collectionCode == "CASTYPE"' caused the original pipe to break prematurely.

To further support the idea that processing in mlr may have caused incomplete processing of:
GBIF.org (01 March 2023) GBIF Occurrence Download https://doi.org/10.15468/dl.pk3trq
I was able to to find CASTPE1652 independently, using
preston cat 'zip:hash://sha256/c8bac8acb28c8524c53589b3a40e322dbbbdadf5689fef2e20266fbf6ddf6b97!/0015281-230224095556074.csv' | cut -f 1,2,3,37,38,39 | pv -l | gzip > gbifid.tsv.gz

and
zcat gbifid.tsv.gz | grep CASTYPE1652

with result yielding:
2238760764	16ec3c7f5-6233-48f6-b36a-06b867edbadd    urn:catalog:CAS:TYPE:1652       CASC    CASTYPE CASTYPE1652
2238760749	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16520	CASC	CASTYPE	CASTYPE16520
2238760442	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16521	CASC	CASTYPE	CASTYPE16521
2238760575	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16522	CASC	CASTYPE	CASTYPE16522
2238760974	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16527	CASC	CASTYPE	CASTYPE16527
2238760477	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16528	CASC	CASTYPE	CASTYPE16528
2238760773	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16529	CASC	CASTYPE	CASTYPE16529
2238760559	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16523	CASC	CASTYPE	CASTYPE16523
2238760930	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16524	CASC	CASTYPE	CASTYPE16524
2238760652	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16525	CASC	CASTYPE	CASTYPE16525
2238760990	6ec3c7f5-6233-48f6-b36a-06b867edbadd	urn:catalog:CAS:TYPE:16526	CASC	CASTYPE	CASTYPE16526

Note that the grep expression also matched catalog number starting with CASTYPE1652 (e.g., CASTYPE16523) in addition to the exact match CASTYPE1652.


Sik-Ho Tang | Review: Trimps-Soushen -- Winner in ILSVRC 2016 (Image Classification).

Sik-Ho Tang. Review: Trimps-Soushen — Winner in ILSVRC 2016 (Image Classification).

Overview
Inthis story, the approach by the winner, Trimps-Soushen, in ILSVRC 2016 classification task, is reviewed. Trimps stands for The Third Research Institute of Ministry of Public Security, or in chinese 公安部三所. In brief, Trimps is the research institute for advancing the technologies for public security in China, which was launched in 1978 at Shanghai. Soushen should be the team name under Trimps, in chinese 搜神. It means god of search, where Sou (搜) means search and Shen (神) means god.
Trimps-Soushen has won several competitions in 2016:

Object Localization: 1st place, 7.71% error.
Object Classification: 1st place, 2.99% error.
Object Detection: 3rd place, 61.82% mAP.
Scene Classification: 3rd place, 10.3% error.
Object Detection from video: 3rd place, 70.97% mAP.

ILSVRC 2016 Classification Ranking http://image-net.org/challenges/LSVRC/2016/results#loc

ILSVRC Classification Results from 2011 to 2016.
Though Trimps-Soushen has the state-of-the-art results on multiple recognition tasks, there is no new innovative technology or novelty by Trimps-Soushen. Maybe due to this reason, they haven’t published any papers or technical reports about it.
Instead, they only shared their results in the ImageNet and COCO joint workshop in 2016 ECCV. And they have some funny facts about the dataset.

Ensemble Using Different Models

ImageNet Classification Errors for Top-10 Difficult Categories.
Trimps-Soushen used the pretrained models from Inception-v3, Inception-v4, Inception-ResNet-v2, Pre-Activation ResNet-200, and Wide ResNet (WRN-68–2) for classification, and found out Top-10 difficult categories as above.

Diverse results are obtained, which means there is no models being dominant for all categories. Each of the models are strong at classifying some categories, but also weak at classifying some categories.
The diversity of models can be used for improving the accuracy. E.g. Boosting!!!

During training, Trimps-Soushen just performed multi-scale augmentation & large mini batch size. During testing, multi-scale + flip are used with dense fusion.

ImageNet Top-5 Error Rate Results.

The validation errors for the 5 best models are from 3.52% to 4.65%.
By ensembling these 5 models (Inception-ResNet-v2 has higher weight), 2.92% validation error for is obtained.
And 2.99% test error is obtained which is the first to obtain under 3% error rate.

Some Findings Based on Top-20 Accuracy

Top-k Accuracy.
Top-k Accuracy is obtained as shown above. When k=20, 99.27% accuracy is obtained. The error rate is smaller than 1%.

Why there are still errors when Top-20 accuracy is used?

Trimp-Soushen has analysed those 1% error images in very detail!!!
They manually analysed 1458 error images from validation set. And roughly 7 categories of errors are obtained as below:

7 Error Categories.

Label May Wrong

Label May Wrong (Maybe it is really a sleeping bag for Hello Kitty? lol).
The ground truth is sleeping bag, but obviously, it is a pencil box!!!!
This is because the ground truths are manually labelled in ImageNet dataset. As ImageNet is a dataset of over 15 millions labeled high-resolution images with around 22,000 categories, and a subset of 1000-categories ImageNet dataset is used for competition, there would be some wrong labels.
There are 211 out of 1458 error images which are “labels May Wrong”, which is about 15.16%.

Multiple Objects (>5)

Multiple Objects (>5) (Which is the main object?).
The above image contains multiple objects (>5). Actually this kind of images is not suitable for ILSVRC classification task. Because in ILSVRC classification task, only one class should be identified for each image.
There are 118 out of 1458 error images which are “Multiple Objects (>5)”, which is about 8.09%.

Non-Obvious Main Object

Non-Obvious Main Object (Please find the paper towel in the image, lol !!)
As only one class should be identified in classification task, the above image does not have one obvious main object in the image. It can be boat or dock. But the ground truth is paper towel.
There are 355 out of 1458 error images which are “Non-Obvious Main Object”, which is about 24.35%.

Confusing Label

Confusing Label (Maybe there is no sunscreen inside, lol.)
The ground truth is sunscreen. This time, the label seems to be correct as the carton saying about SPF30. But the task would become to understand the meaning of the text on the carton, which is going too far from the original objective of recognizing objects based on shapes and colors.
There are 206 out of 1458 error images which are “Confusing Label”, which is about 14.13%.

Require semantic understanding!!!

Fine-Grained Label

Fine-Grained Label.
The ground truth is correct. Both bolete and stinkhorn are the types of fungal. Indeed, this type of label is even difficult for human to identify.
There are 258 out of 1458 error images which are “Fine-Grained Label”, which is about 17.70%.
The network can improve this category.

Partial Object

Partial Object
The image may only contain a part of the object, which is hard to recognize. Maybe the image can be better if it is zoomed out with multiple tables and chairs, looking like a restaurant.
There are 66 out of 1458 error images which are “Partial Object”, which is about 4.53%.

Therefore, the accuracy is hard to be improved by 1%.

Region Fusion (Image Localization)

Region Fusion for Image Localization.
To localize the top-5 predicted labels within the image, Faster R-CNN architecture using multiple models is used. Multiple models are used to generate region proposals via the region proposal network (RPN) in Faster R-CNN. Then based on the top-5 classification predicted labels, to perform the localization prediction.

Image Localization Top-5 Validation Error Results.

Previously state-of-the-art approaches: 8.51% to 9.27% error are obtained.
Ensemble all: By fusion all approaches together, 7.58% error is achieved.
Ensemble all but without one model: Only 7.75 to 7.93% error is obtained.

Thus, diversity between model is important and contribute large improvement on prediction accuracy.

ILSVRC Localization Top-5 Test Error Results from 2012 to 2016.

Multi-Model Fusion for Other Tasks (Scene Classification / Object Detection / Object Detection from Video)
Scene Classification

Multi-Scale & Multi-Model Fusion for Scene Classification.
Improved multi-scale approach is used by concatenating the results instead of just adding the results together at the end of network.
On top of using multi-scale inputs for prediction using the same model, two trained models (I believe using the same model network) are used by concatenating the results and goes through FC and softmax. 10.80% validation error is obtained.
By using 7 * two models, 10.39% validation error and 10.42% test error are obtained.

Scene Classification Top-5 Test Error Results.
With the model using external dataset, Places2, for pretraining as well, 10.3% top-5 test error is obtained which got 3rd place in scene classification.

Object Detection
Similar to image localization, Faster R-CNN architecture is used with multi-model fusion.

Object Detection mAP Results.
61.82% mAP is obtained.

Object Detection from Video

Object Detection from Video mAP Results.
Optical flow guided motion prediction is also used to reduce false negative detection. 70.97% mAP is obtained.

Due to the model diversity, model fusion is effective. By using model fusion, Trimps-Soushen outperforms ResNeXt and PolyNet and got 1st place in image classification. And model fusion is also successfully applied to other tasks.

This means that except the innovative optimization or novel design of network architecture, other technical stuffs such as multi-model fusion can also help to improve the accuracy a lot.

On the other hand, if Trimps-Soushen uses ResNeXt and PolyNet for model fusion, perhaps the error could be further reduced, since ResNeXt and PolyNet obtain higher accuracy compared with those models used for model fusion.

References

[2016 ECCV] [Trimps-Soushen] (Slides Only) Good Practices for Deep Feature Fusion.


Adding Semantic Release

:tada: This PR is included in version 1.0.1 :tada:
Your semantic-release bot :package::rocket:


Warning: could not generate index.html (using provided install instructions)

I a developer were to follow the instructions provided in the README.md they would recieve the following warning when generating a hologram-based style guide:
$ hologram
Custom markdown renderer Cortanamarkdownrenderer loaded.
Warning: Could not generate index.html, there was no content generated for the category README.
Build completed. (-:

This is because the setting:
# Mark which category should be the index page
# Alternatively, you may have an index.md in the documentation assets
# folder instead of specifying this config.
index: README

seems to declare a category that will drive the index rather than the name of file.

I'm having the same issue here with Hologram 1.2.0 and Cortana 1.2.8:
$ hologram -c hologram_config.yml
Custom markdown renderer Cortanamarkdownrenderer loaded.
Warning: Could not generate index.html, there was no content generated for the category README.
Build completed. (-:

I have a README.md file containing markdown in this directory and the following in my hologram_config.yml:
# To have a custom index page build with your README.md
index: README

I see that the Cortana-example has been built quite some time ago. How can I build and test this example locally, where do I find the source with the corresponding config.yml?

+1 Having the same issue.

I'm using Hologram 1.3.1 and Cortana 1.2.8. Creating the README.md in my source directory prevented this warning from popping up, so it seems like this is fixed now.

I don't think this is fixed. I'm having the same issues as mentioned above. If I set a valid category as the index page it generates the index.html file ok. It fails to work when i try to use a readme.md file.

In my hologram_config.ml file
source: ./assets/css/sass
To have a custom index page build with your README.md
index: README
/assets/css/sass/README.md contains some markdown and it's generating the style guide for me. I don't know if maybe your source file isn't picking up from the right place? What versions are you using?

Hi, I have the same problem :(

@cosmicdreams create a index.md in your source folder and remove the index param in hologram.yml
my hologram ex:
source: ./assets/sass/
My path = assets/sass/index.md


Add linear scale in redemtion

We need to also make redeemptions pooled

Add the liner scale into the pool instead


fix(curriculum): typos in Nutrition Label section

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Step 25 Removed comma
Step 53 Capitalized sodium
Note: FYI, I only changed 2 files but the console showed 3 files changed. I'm not sure if that is an issue or not.

What is tream? I only opened and changed the two lesson files. However, I used the git command git fetch upstream but once again I wasn't on the main branch. I was in my branch fix/rothkopainting-typos. Could that have anything to do with it? Regardless, do I need to do something?

Check the Files changed tab, I'm not sure what it is and how it made to the commit (although image shows it as a part of commit), but that file will need to be removed from PR. It can be done as an additional commit.

So if I did git reset HEAD that wouldn't do it? And then the command to reset the add but I can't remember that one. And then instead of doing git add ., could I add just add the changes with the file names like step-25.md?
Do you even recognize that code as something from freeCodeCamp? Do you have a link on how to remove that via another commit because I do not know how to do that?


Support the Spec/API/Design-first approach

This project currently only supports the Code-first approach, in other words, it generates the OpenAPI spec from your code and keeps the spec in sync with your code afterward. While the Spec-first approach allows you to write the spec first, when every detail of the API is discussed and confirmed, you start to code based on the spec. Both approaches have their own pros and cons. Although Code-first seems quite convenient and popular, I still hope to support the Spec-first approach for APIFlask, and it's the recommended approach by the OpenAPI initiative.
Ideas
A good starting point is to support for a Semi-Spec-first approach:

Provide a codegen tool, generates the skeleton code (routes, schemas, models, etc.) from an existing spec.
After the generation, turns to the Code-first approach. In this way, the spec can easily keep sync with the code.

After this, we can start to think of supporting the real Spec-first workflow. TBH, I don't quite familiar with the Spec-first approach, especially:

How to keep the code and spec in sync? If a user uses the codegen tool, after the code is generated, if we change a field name in the spec, then we have to update the code manually?
If the validation is based on the spec and JSON schema, how to add pre/post procession hooks for the input/output data?
What's the ideal workflow?
What and how much the framework should get involved?

Before these happen, maybe we can recommend the user to use Connexion in the docs since it's now back to the maintenance status (https://github.com/zalando/connexion/issues/1365).
Resources
See a relevant discussion here.
Learning resources checklist:

[ ] Contract-First API Development Using The OpenAPI Specification (Swagger) - PyCon 2017
[ ] Connexion's source code
[ ] Connexion's documentation

Hey @greyli, just came across this from the OpenAPI Generating page. It's been three years since you've created the issue, but I'm still quite interested in APIFlask supporting this. Wondering if your views on this have changed since you've created the post?


undefined method `photos_metadata' for nil:NilClass

<%= f.input :photos, as: :attachinary %>
has_attachments :photos, :maximum => 3
Rails 4 app with attachinary & cloudinary.
Help me plz

Do you have the following in application.rb?

require "attachinary/orm/active_record"


"--with-provider2" flag apparently needed on barge. If we really do need it, fix docs. If no, deprecate the need

Problem
Until recently, we could run unit tests with barge without any special flags. But I ran into issues, reported eg in #162. Calina noted that in order to give any chance of working, I needed it to have "--with-provider2" flag.
To be precise, the current "recommendation" to test things locally is:

Follow steps 1-5 of developers.md, choosing barge steps; except in step 2, make sure barge is clean, and run with provider 2 as follows

run docker system prune -a --volumes    # cleanup the old containers
./start_ocean.sh  --with-provider2   # run barge with provider on

This is not not currently in our READMEs (quickstarts and developers.md). Therefore it's a bug because people importing ocean-lib or developers working on ocean.py will fail.
And it's a pretty weird flag.
@kremalicious comment: "I'm also confused about that new --with-provider2. It seems both libraries have this now set in Travis since yesterday. If this flag is required to run anything successful against barge, it should be a default over there"
Describe the solution you'd like
Do we really need that flag?

If no, then let's deprecate the need for it.
If yes, then let's improve the documentation

BTW this could be considered an aquarius bug, not ocean.py. I wrote it here since I'm approaching this from an ocean.py user perspective.

yup, basically 2 solutions, either:

make --with-provider2 a default option in barge, effectively deprecating that (undocumented) option there
update all repo READMEs where barge is a requirement for running tests with that option, and document that option in barge README

We should probably discuss this over on barge, but from what I can see the --with-provider2 is only there to workaround an issue which only happens when using barge to run integration tests against it. Since half of barge's use case is to run tests against it, I would be in favor of 1., defaulting to that option. If we opt for 2. then a better name reflecting the use case for that option might be better, like --testMode or something

Update: so that at least people using developers.md aren't stranded, I added --with-provider2 to its instructions as part of commit https://github.com/oceanprotocol/ocean.py/commit/da350f3ab4d1df20cfe85220d97c343f78c30322 (towards #113 better dev instructions).

ok, then it sounds like we want to go with option 2, updating barge & libraries READMEs. Would still advocate for naming that option based on what it is intended to be used for, like --test.
Then we can document it maybe like so:
`--test` | For running Compute-to-Data integration tests. Will start an additional `provider` where an asset is served by provider1 and an algorithm served by provider2

resolution: keep this option as is and document it in Barge.

document it in Barge

New issue created for that: barge#233
So we can close this issue.


Unclear instruction?

In the README.md under "Libraries" it says:

"To help the language server to find external libraries and parse their headers you need to unarchive the *.zip files in the .internal/lib/ folder."

Where is the  .internal/lib/ folder spoken of in this instruction?

.internal/lib/ in the project root folder. This folder appears after the external libraries are installed.
If there are no external libraries in the project, there is no .internal/lib/ folder.


Better hint for non-zero em length

Closes #4304

Probably missed this:

maybe we should also flip the order of the hints? ignoring is generally not as advisable.

@laurmaedje yes sorry, went over it a bit too quickly, done ;)

Thanks!


allow promoting a release to a different channel

this technically works, but i think resetting the installation cursor when a new license is uploaded with a different channel id is a cleaner approach.

also, would this still allow uploading airgap bundles from the previous channel?


Updated Windows previewer with UASTC support

Updated Windows previewer with UASTC support. Tested with both ETCS and UASTC files.

Thank you!


feat: Ignore DDL query strings on temporary relations

Fixes #71
Temporary relations are not replicated, so we shouldn't report a query string for those.
This commit adds infrastructure to detect whether a given command is for a temporary relation or not, using the same internal function as upstream postgres, to guarantee that the detection will be identical.  The detection is not always easy as unqualified relation name depends on the search_path and various rules to detect what the final relation will be.  For instance, persistent view creation are automatically switched to temporary views if they reference temporary objects.
Finally, DROP commands are even trickier as they can reference both persistent and temporary relation.  It means that a new query has to be generated to ignore any temporary relation if any.
Some other DDL may need to be similarly ignored, like GRANT / REVOKE or COMMENT ON, but it seems very unlikely to use such commands on temporary tables.  So they are not handled by this commit, to avoid too much code to maintain. Additional checks can be added later on if needed.

(I forgot to git add the new tests, fixed now).

Wow. This PR is really impressive. @KennyChenFight  and I definitely can't figure out all this in such a short time. Thanks @rjuju!

thanks a lot 🙏🙏 I'm glad I can help with the project!

Thanks again! What a great work!

Hi @rjuju, we also sent you a GitHub Org invitation so you can have direct write access to this repo. 😊

sorry, I somehow missed that message.
thanks a lot, I'm really honored to be part of the org!

@rjuju
Hi, welcome to join our slack!


add missing type parameter in Generic.Aux scaladoc

Not completely sure it was not intended, let me know.

Well spotted ... thanks!


vim motion doesn't work while typing in Japanese...

The plugin doesn't let me enter an insert mode from normal mode when I'm typing in Japanese. It would be huge for me if it could be done...
I'm using it on M1 MBP with Sonoma 14.5.

Unfortunately, it is by design. You can reproduce the issue here: https://codemirror.net/5/demo/vim.html
I hope it will be solved when migrating to CM6, which is planned in the future.

I c, noted. Thank you for getting back on this!  really hope so too...!


color stop positions

I'm not used to how PostCSS works under the hood so this might be a silly question.
Is it possible this plugin could support a subset of the positional arguments for color stops eg:
linear-gradient(
  to top,
  black 0%,
  ease-in-out,
  transparent 15%,
  linear,
  transparent 100%
);

I understand that this probably not possible to polyfill the entire set of positional arguments (calc, mixing %/px etc.) but surely just linearly interpolating between percentages should be possible given the output css uses percentages?
I came across this limitation while using mask-image with the linear gradient as a fade-out on the bottom of an image. This problem would usually be easily alleviated with background-size but, unfortunately, mask-image-size works differently such in such a way that anywhere not covered by mask-image is masked out. So the image fades nicely then just becomes transparent for the most part.

I am want this feature too

I ran into this same problem with mask-image, and I seem to have solved it by using multiple gradients, one that's a solid black and another, with a fixed height, as the fade, along with a few of the other mask-* properties to place and composite them. I compile from Sass before running PostCSS, but you should be able to adapt this to vanilla CSS easily:
  // We have to define the linear-gradient with easing separately as a custom
  // property because either PostCSS or postcss-easing-gradients replaces the
  // comma in "linear-gradient(black, black)" with a semicolon, breaking the
  // value. 🤦
  --gradient-mask: linear-gradient(
    transparent,
    #{$easing-decelerate},
    black
  );

  $fade-size: 4em;

  // The first linear-gradient is used as a solid colour that spans the full
  // width and height, while the second gradient is the actual fade out. This
  // allows us to set a specific size for the fade out (see $fade-size) even
  // though postcss-easing-gradients doesn't yet support colour stop positions.
  // This works because of the mask-composite values, which cause the solid
  // colour gradient (the first) to have the alpha of the second (the fade)
  // substracted from it, so that it doesn't overwrite the fade portion.
  //
  // @see https://developer.mozilla.org/en-US/docs/Web/CSS/mask-composite
  //   The standardized composite property, supported by Firefox/Gecko and Edge
  //   18 (pre-Chromium) as of 2019-11-29.
  //
  // @see https://developer.mozilla.org/en-US/docs/Web/CSS/-webkit-mask-composite
  //   The non-standard property required for Chrome/Blink, Opera, Android
  //   WebView, and Safari/WebKit as of 2019-11-29.
  //
  mask-image:     linear-gradient(black, black), var(--gradient-mask);
  mask-size:      100%, 100% $fade-size;
  mask-repeat:    no-repeat;
  mask-position:  center, center bottom;
  -webkit-mask-composite: xor, source-over;
  mask-composite: subtract, add;


[PROFILE REVIEW]

GitHub Profile Link:-
https://github.com/ozboware

Hey @ozboware
Make the following changes to your profile:

README file is too long; make it shorter
Add your projects in the pinned repo section and remove the screenshots to make it look good
An about section and a banner at the top is recommended

💡 Tip: Don't follow so many 3.6k people. It's a lot.


Update project to use latest version of axum

The project is built with Axum-v0.5.16, update the project and peer dependencies to use the latest Axum version

Implement axum-v1.0


Restructuring Deployment interfaces and related

this is a branch off https://github.com/Azure/azure-sdk-for-java/pull/905

Can one of the admins verify this patch?


sets default as array if the attr is an Array

This PR sets an attribute's default to array if the type is Array and has no other default defined.
This solves the following error when working with nested_attributes when using fields_for with a child_index:
undefined method `stringify_keys' for "Value":String.

To reproduce the error use this example https://www.stimulus-components.com/docs/stimulus-rails-nested-form/
This error occurs because an attr_json defined as array:true is created as nil and does not have the to_ary method which is used by Rails to create the child_index
https://github.com/rails/rails/blob/436207aa2011463e49feb6310fa144bb0a96bf0c/actionview/lib/action_view/helpers/form_helper.rb#L2683
This can also be solved by setting the attr default to default: []

Hi, thanks for trying out attr_json and getting in touch!
I worry that everyone might not want this behavior. After this PR, if you had an array attribute, saving it would always save an empty array [] in the JSON in the db, when previously it would be possible to store nil and/or leave the key out entirely.
Those extra bytes will take up space in your database, and everyone might not want them.  If we were to do it, we'd also have to do it on a major release of attr_json, since it's a potentially backwards incompatible change people might not be expecting otherwise.
(Also, it would need some rspec tests with the PR if we were otherwise interested in merging it!)
As you note, this can already be solved by setting the default yourself on every attribute.
I am glad there is a way to get it to work, but I can see why this is annoying to have to set that on every attribute. If we wanted to improve this, I think the way would be to add a configuration to attr_json that was off by default, but which you could turn on, to default all arrays to empty array. I guess you'd still need to set it in every record, so I'm not sure how helpful it would be.
What do you think?
If other people have this issue, I'd maybe consider this in a future attr_json 2.0, but we can't do it now in 1.0.

I'm working on this for a possible attr_json 2.x release, since I think it is backwards incompat.
It is ending up trickier than expected.  But also... it seems to be legit breaking some accepts_nested_attributes_for behavior, in ways I don't understand. I'm trying to investigate.

OK, I'm working on getting this into an upcoming 2.x release!
It required a lot of changes to specs, and actually ended up bumping into a weird existing bug triggered on some edge cases. #160
@g13ydson , I think you might want to do default: -> { [] } instead of default: [] -- to avoid accidentally re-using the same array in all objects. As with this frequent ruby bug. But maybe it ends up not an issue here?
I'm excited that you were using attr_json with stimulus, and that it was mostly working -- I haven't used stimulus yet myself. Very curious to learn more about this, and if you are still doing it and it's still working for you!


Server Error ReferenceError: Cannot access 'TwitterApiReadWrite' before initialization  [bug]

Describe the bug
I am getting this error anytime I just import the library & do a simple call: e.g.
let res = await client.v2.tweet("Hello");

Version

Node.js version: 20.1
Lib version: 1.16.4

Additional context
All of this worked really well before, not sure what's changed? My codebase is pretty much the same.

did you find the issue with this?

up

I'm getting same issue

    const client = new TwitterApi({
      appKey: env.TWITTER_CONSUMER_KEY,
      appSecret: env.TWITTER_CONSUMER_SECRET,
      accessToken: env.TWITTER_ACCESS_TOKEN,
      accessSecret: env.TWITTER_ACCESS_TOKEN_SECRET,
    })

    const rwClient = client.readWrite

    const tweetText = endent`Hello World!`

    try {
      const result = await rwClient.v2.tweet(tweetText)
      console.log('Tweet posted successfully:', result)
    } catch (error) {
      console.error('Error posting tweet:', error)
    }

Same here, using next.js server actions.
const twitterClient = new TwitterApi(twitterAppBearerToken);

Hello,
We're not maintaining actively this library anymore, however, I accept PRs.
Any idea in what version it started though?

Hi @PLhery,
Damn, that’s sad. PR about improvement?
I’ll inspect and see when it started.

getting the same issue

Also running into this with my NextJS project

Running into same problem with nextjs 14 server actions. Any solutions?

Running into same problem with nextjs 14 server actions. Any solutions?

I have found the issue. It's a problem with bundling. Specifically turbo. If you're using nextjs and you're builing with --turbo, simply remove that.
 "dev": "next dev --turbo" should become  "dev": "next dev",
const postTweet2 = async (text: string) => {
  const userClient = new TwitterApi({
    appKey: process.env.TWITTER_CONSUMER_KEY!,
    appSecret: process.env.TWITTER_CONSUMER_SECRET!,
    accessToken: process.env.TWITTER_ACCESS_TOKEN!,
    accessSecret: process.env.TWITTER_ACCESS_TOKEN_SECRET!,
  });

  try {
    const tw = userClient;

    const tweet = await tw.v2.tweet(text);
  } catch (error) {
    console.error("Error posting tweet:", error);
    throw error;
  }
};

Describe the bug I am getting this error anytime I just import the library & do a simple call: e.g.
let res = await client.v2.tweet("Hello");

Version

Node.js version: 20.1
Lib version: 1.16.4

Additional context All of this worked really well before, not sure what's changed? My codebase is pretty much the same.

Just to confirm @ninsau's solution worked for me! Thanks for the suggestion

I can confirm in nextjs 15 using turbopack causes that issue. Removing --turbopack did the trick. Thank you @ninsau

It looks like there's an actual fix in PR but needs to TS updates : https://github.com/PLhery/node-twitter-api-v2/pull/544
Who's the author here?  All i see is the integration bot

Actually, I asked an AI SWE agent (devin) to fix the issue see if it leads to somewhere. But it did not do the right thing, closing the PR.
I think for now not using --turbo is the best fix, it seems to be mostly an issue from their side.

anyone figure out a way to get it to work while not losing turbo?

add "twitter-api-v2" to the serverExternalPackages array in your next.config.ts file


Windows App Certification Kit

I'm developing a Cordova App using VS2015.
I'm able to build in debug and release modes but when I try to create app packeges for windows I get the following errors:
Error Found: The binary analyzer test detected the following errors:
File c:\program files\windowsapps\38c6d89a.dietmedprofissional_<IP_ADDRESS>_x86__3bezmf8a4wq6j\SQLite3.UWP.dll has failed the SafeSEHCheck check.
File c:\program files\windowsapps\38c6d89a.dietmedprofissional_<IP_ADDRESS>_x86__3bezmf8a4wq6j\SQLite3.UWP.dll has failed the WXCheck check.
and:
Error Found: The supported APIs test detected the following errors:
API _CrtDbgReport in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _CrtDbgReportW in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API __stdio_common_vsprintf_s in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _cexit in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _configure_narrow_argv in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _crt_at_quick_exit in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _crt_atexit in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _except1 in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _execute_onexit_table in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
API _free_dbg in ucrtbased.dll is not supported for this application type. SQLite3.UWP.dll calls this API.
Please help.
Thanks in advance.

I don't ship an assembly called SQLite3.UWP.dll
Are you also using sqlite-net-pcl ?  See praeclarum/sqlite-net
My guess is that maybe the UWP assembly in sqlite-net-pcl is perhaps not built with release mode.  That guess is partly based on a vague recollection of seeing this problem before, but if there is a github issue for it, I can't find it right now.


Question : how to use custom gstreamer graph as video source?

Hi @alfredh ,
I'm trying to build a video intercom based on IP camera, baresip SIP client and asterisk server.
Stream from IP camera can be proceeded by GStreamer with graph:
rtspsrc name=rtspsrc latency=0 tcp-timeout=2000000 is-live=TRUE<EMAIL_ADDRESS>! progressreport update-freq=60 do-query=true ! rtph264depay name=depay !  video/x-h264,stream-format=byte-stream,profile=baseline,alignment=nal !  appsink name=sink
Output sink contains H264 payload with suitable bitrate/framerate (setings at WEB interface of camera). I need to have a video call with "sendonly" flag in SDP for video RTP and origin stream from sink without any re-encoding (just only encapsulating to RTP protocol) from baresip.
Is it possible to do?
Thanks a lot!

Hi,
there is a gstreamer module for video encoding:
https://github.com/alfredh/baresip/tree/master/modules/gst_video1
this module takes the input frame from inside the baresip stack,
and encodes it using gstreamer.
so it is most likely not possible to do what you suggest, unless you do some hacking.
the avformat.so module can take an RTSP source via network and use it as a
video source device, the module will also decode the incoming video stream.
baresip will re-encode the video stream to the video codec selected via the
SDP negotiation to the peer. (i.e. transcoding).


[objc] Give the processor the ability to add external/additional code

Process the type queue while processing the assembly one (not after);
Add AddExtraType so it's possible to add external/additional code to
be included (e.g. issues #175 and #176);
Make sure it's safe to call GetProcessedType (or throw)

references:

https://github.com/mono/Embeddinator-4000/issues/175
https://github.com/mono/Embeddinator-4000/issues/176

Build and tests passed.


Define custom eventWrapper

Hi,
I am trying to define a custom event wrapper but when i use mine es:
const customWrapper = ({event}) => <AppointmentSummary  title={event.title} start={event.start} end={event.end}/>
I lose the positioning of the event in the calendar

You are using the wrapper component incorrectly. It is a wrapper not a replacement. you need to render this.props.children which is the event.

Thanks, i am still not sure i am using it right :
 render() {         const { event, children } = this.props;         return (<div> className={children.props.className} style={...children.props.style}>             <AppointmentSummary {...event} />         </div>);     }

This is what i got, if there is a correct way I would be glad if you could show it to me
 MyWrapperComponent ...
 render() {
        const { event, children } = this.props;
        return (<div> className={children.props.className} style={...children.props.style}>
            <AppointmentSummary {...event} />
        </div>);
    }

I need to pass the className (.rbc-event) since it has "position: absolute" while the inline style contains the coordinate informations.
It feels kinda hacky?! Also doing so i cant select the event when using the onSelectedEvent and selectable props.

You just can't  use this prop like that. You don't want the wrapper component anyway you want the event component prop. See the custom render example

I've seen the example, but it just renders the event component while keeping the wrapper, if I do that I cant change the fact that the wrapper has a title with the hour, a blue rectangle etc.
To be honest i feel that the custom rendering part is not well document, whatever I found has been trough issues here.
Maybe its just me but if you could provide a small example I am sure it would helpful for anyone trying this library.
Thanks for your time

The issue is isn't an api for doing what you are trying to do. The Wrapper components are really for addon authors, who need to add a new component around the original event component, you are trying to replace the even component completely, which isn't supported. There is intentionally not a ton of control given over these components because the calendar needs to control their position and style fairly percisely in order to lay them out. You can change the content of the even with the event Component and the styling of the event shape with css or eventPropGetter

Ok thanks for the explanation, for anyone that eventually will encounter this I resorted to using eventPropGetter for overriding the available parts of the wrapper style and defining a custom event component


My Intellisene stopped working unexpectedly. I've been trying to solve this for 6hrs and still I was unable to. This happened while working on my Nextjs project. It's not showing auto complete anymore.

/duplicate of #139447


Question regarding original sources of emojis

Dear twemoji contributors,
we would like to package twemoji for Debian, which requires that we need to know about the original sources.
This commit:
https://github.com/twitter/twemoji/commit/d008619cc043b65bc6283f2b0d882b8d088811c2
is titled "Remove the AI assets, as this is a proprietary format and the SVGs should suffice for editing".
Does this mean the .ai files are no longer used, and the SVGs are the source (which are edited by AI, inkscape or whatever)?
Many Thanks and Best Regards,
Felix

Reply from Justine De Caires<EMAIL_ADDRESS>The original sources are indeed the SVG files. When we update the package
with new emojis, I'm given SVG files by the designers, so there aren't any
AI files to be had anymore.


ResolverResult not setting Value

Describe the bug
The constructor of the ResolverResult class does not initialize Value member correctly.
To Reproduce

var rr = new ResolverResult(1);
rr.Value is null

Constructor currently initializes Value to "default" of the type.

@seblescure I fixed the bug and wrote some tests. I will create a new patch release tonight.

@seblescure fix is included in version 4.2

Thanks for the quick fix :)


Port tests from azure-sdk-for-node/runtime/ms-rest-azure

@daschult should this be considered done?

Probably. We have a decent test coverage now, so I think you can close it.


feature/#575 pandas 2.0 support

#575
BLOCKED: due to modin not yet officially release version 0.23, which also contains pandas 2.0.2
TODO:

[x] Update based on changes in pandas 2.0.2
[ ] update setup.py and pipfile to support new modin release (also contains new pandas version)

@FabienLelaquais
The plan is to wait for modin to release their newest version (0.23.0) and update our Pipfile and setup.py file then.
You can expect to see the changes in:

Pipfile -> modin = {extras = ["dask"], version = "==0.23.0"}
setup.py -> "modin[dask]>=0.23.0,<1.0",

After changing the modin version to use 0.23.0rc0, a test fails to read an excel file.
I have created this bug report on modin project: https://github.com/modin-project/modin/issues/6338


Question on how to match all package.json not in node_modules

I have several nested package.json files, and I'd like to find all the ones that are not under a node_modules directory.
The following seems like it should work, but it returns things like demo/app/plugins/content/node_modules/extglob/package.json
glob("**/package.json", {ignore: ['node_modules/']}, (err, files) => {
        console.log('globe error', err);
        console.log('files', files);
});

I've also tried using the !(...) pattern, but that doesn't return everything:
"**!(node_modules)/package.json"
Any suggestions on what would work?

Okay, got it, will leave answer here in case anyone trying to do something similar:
glob("**/package.json", {ignore: ['**/node_modules/**']}, (err, files) => {
        console.log('globe error', err);
        console.log('files', files);
});


feat(crypto): add BLS12-381 curve

curve based on blst.
The bls12381 build tag was added. Also, note that BLS requires cgo.
Co-authored by: @tac0turtle

PR checklist

[x] Tests written/updated
[x] Changelog entry added in .changelog (we use unclog to manage our changelog)
[ ] Updated relevant documentation (docs/ or spec/) and code comments
[x] Title follows the Conventional Commits spec

Thanks @melekes I'll take a look as soon as I can

@melekes there are non cgo versions as well just fyi, we should support both

@melekes there are non cgo versions as well just fyi, we should support both

@itsdevbear, do you have a Github link? @tac0turtle, if there are non-CGO versions, why did you choose a CGO version?

I think the one in geth
https://github.com/ethereum/go-ethereum/tree/master/crypto/bls12381
is.
Otherwise I can dig it up in the morning just in bed
I imagine the CGO one is more performant

either way, usually good to support multiple implementations to guard against vulns

This is used in geth:
https://github.com/protolambda/bls12-381-util

Protolambda's stuff is good

Cgo is more performant. Since there is no aggreagtion in comet, yet the performance overhead will be high for doing individual signatures.

I added a few comments, some of them important. Sorry for arriving late to this review


Added integration test that would close #102

This integration test would close #102.

@jeffkala - I don't know if this is feasible or not.  The CI pipeline keeps failing for reasons mentioned here.  I could see this being an intermittent failure to the pipeline going forward with it's dependency on the celery worker.  Thoughts?

What if we moved them all to requests mock?

What if we moved them all to requests mock?

I think you're thinking of a unit test. The request was for an integration test.


Firefox plugin gets a bit excited about transient changes

I'm finding that the plugin seems to tell me every few hours that my fingerprint has changed, but all that seems to differ is 'adBlock' status.  Now, I have it set so some sites are not blocked while most are, and I wonder whether it is looking a bit too often or something.
If nothing else, perhaps recognising that I have two identical fingerprints that I seem to be toggling between might be advised.

Thanks a lot for reporting this to us! I remember that I observed the same thing at some point but I forgot to look into it. We are currently preparing a new version of AmIUnique and I'll add that to the todo list of things to improve.


Angular Amplify Frontend Build fails when trying to deploy

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
dx01a9f9736el
Region
us-east-1
Amplify Console feature
Not applicable
Describe the bug
I setup amplify deployment with CI/CD for my angular application. amplify backend successfully deployed to amplify console, but frontend gives following error message.
/codebuild/output/src806740382/src/rov-web/amplify.sh: line 18: unexpected EOF while looking for matching `"'
/codebuild/output/src806740382/src/rov-web/amplify.sh: line 19: syntax error: unexpected end of file
Please find attached log file and the amplify.yml file I used for this deployment.
I tried to fix this but still couldn't find anything. Please help me to fix this. Thank you
log (10).txt
Expected behavior
App should be able to deploy to amplify console.
Reproduction steps

install amplify with amplify init
Added host using amplify hosting add
configure CI/CD deployment
commit the code to AWS code commit

Build Settings
version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    # IMPORTANT - Please verify your build commands
    build:
      - npm run build
  artifacts:
    # IMPORTANT - Please verify your build output directory
    baseDirectory: dist/ng-ace
    files:
      - '**/*'
  cache:
    paths:
    - node_modules/**/*

Additional information
No response

Hi @pralivera 👋🏽, can you check to make sure your builds run successfully on your local machine? Usually the unexpected EOF while looking for matching `"' means that you have a string with either a missing " or an extra " somewhere in your codebase.

I was able to deploy it to s3 bucket.
On Sat, 6 Nov 2021, 01:51 hloriana, @.***> wrote:

Hi @pralivera https://github.com/pralivera 👋🏽, can you check to make
sure your builds run successfully on your local machine? Usually the unexpected
EOF while looking for matching `"' means that you have a string with
either a missing " or an extra " somewhere in your codebase.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws-amplify/amplify-console/issues/2368#issuecomment-962196171,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABQMNEN2JVIE6SNXXZVH5KTUKRDFPANCNFSM5HOTCUGA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@hloriana I was able to deploy it to s3 bucket.

@pralivera Awesome! Feel free to submit another issue if you run into another blocker.


fix(store/cache): unstick cache metrics names

Metrics names were not properly delimited. Example:
eds_cacherecent_evicted_counter_total

Codecov Report
Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 46.29%. Comparing base (2469e7a) to head (74eebfb).
Report is 340 commits behind head on main.

Files with missing lines
Patch %
Lines

store/cache/metrics.go
0.00%
4 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3857      +/-   ##
==========================================
+ Coverage   44.83%   46.29%   +1.46%
==========================================
  Files         265      314      +49
  Lines       14620    18142    +3522
==========================================
+ Hits         6555     8399    +1844
- Misses       7313     8719    +1406
- Partials      752     1024     +272

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[KYUUBI #1022] Add EventManager with basic EngineStatusStore

Why are the changes needed?

For more detail, please go to https://github.com/apache/incubator-kyuubi/issues/981
EngineStatusStore helps to push events to listener bus
EngineStatusStore is a memory store that tracking the number of statements and sessions, it provides:

stores all elements, and sorted by startTimestamp.
cleanup the last elements when reach a certain threshold.

How was this patch tested?

[x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

cc @yaooqinn, realized a single memory store called EngineStatusStore.
It's ok to rework with ElementTrackingStore. https://github.com/apache/incubator-kyuubi/issues/981#issuecomment-912440645

Thanks @cfmcgrady @yaooqinn, make the current patch to draft, starting rework this patch by using ElementTrackingStore.

Codecov Report

Merging #1023 (48b1982) into master (05fc1c0) will decrease coverage by 0.68%.
The diff coverage is 68.71%.

@@             Coverage Diff              @@
##             master    #1023      +/-   ##
============================================
- Coverage     80.67%   79.99%   -0.69%
  Complexity       14       14
============================================
  Files           159      162       +3
  Lines          5837     6208     +371
  Branches        657      701      +44
============================================
+ Hits           4709     4966     +257
- Misses          734      820      +86
- Partials        394      422      +28

Impacted Files
Coverage Δ

...g/apache/spark/kyuubi/SparkSQLEngineListener.scala
86.53% <ø> (ø)

...n/scala/org/apache/kyuubi/KyuubiSQLException.scala
91.56% <ø> (+5.20%)
:arrow_up:

...ommon/src/main/scala/org/apache/kyuubi/Utils.scala
64.44% <0.00%> (-3.80%)
:arrow_down:

...uubi/engine/spark/operation/ExecuteStatement.scala
80.39% <50.00%> (-8.64%)
:arrow_down:

.../apache/spark/kyuubi/ui/EngineAppStatusStore.scala
58.00% <58.00%> (ø)

...cala/org/apache/spark/kyuubi/ui/EventManager.scala
61.22% <61.22%> (ø)

...org/apache/spark/kyuubi/EngineStatusListener.scala
64.57% <64.57%> (ø)

...kyuubi/engine/spark/operation/SparkOperation.scala
68.25% <66.66%> (-0.60%)
:arrow_down:

...rg/apache/kyuubi/engine/spark/SparkSQLEngine.scala
72.60% <100.00%> (+1.17%)
:arrow_up:

...he/kyuubi/engine/spark/operation/GetCatalogs.scala
92.85% <100.00%> (+9.52%)
:arrow_up:

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05fc1c0...48b1982. Read the comment docs.

How about we reuse the current event logging service and only introduce the ElementTrackingStore or something else as an in-memory implementation?

How about we reuse the current event logging service

It's not a right way to reuse event logging service.

We should push events to SparkListenerBus, if not, the spark history server will not receive these custom events.
It's better to let different listeners do their own independent things

and only introduce the ElementTrackingStore or something else as an in-memory implementation?

In fact, using ElementTrackingStore is the least expensive, if we create something as an in-memory implementation will spend more time. I had tried these two ways.

We should push events to SparkListenerBus, if not, the spark history server will not receive these custom events.

can we check SparkHistoryEventLogger?

How about we reuse the current event logging service and only introduce the ElementTrackingStore or something else as an in-memory implementation?

+1

hi @yaooqinn, I can not understand why kyuubi need SparkHistoryEventLogger? it seems duplicated to spark.eventLog.enabled=true.

Looked into the logger service, the loggers are executed sequentially, and the loggers are called synchronously.

And checked the LiveListenerBus, it is an asynchronous listener bus for Spark events. SPARK-20863

hi @yaooqinn, I can not understand why kyuubi need SparkHistoryEventLogger?

SparkHistoryEventLogger is used to log all Kyuubi's events to the same file of spark's event log. We also support other destinations, like Kyuubi's own JSON log store, JDBC(planned), etc..

it seems duplicated to spark.eventLog.enabled=true.

It's not. it follows spark.eventLog.enabled when logging. We just don't push the events to the listener bus but call the event log listener directly.

And, the current design of logger service is not clear to me. As we know a service means that it should listen to some port and can start / stop.

Hmm... do not go too far with the spark's listeners and the listener bus if you are not understanding what you are doing here.
IIUC, you want everything to work with spark's listeners and the listener bus, this is not what we want.  We are a Spark application BUT not only a spark application that is limited to spark's functionalities.
What we want is only a size-limit buffer (using ElementTrackingStore is just an option) that holds all the current Kyuubi events in memory. Then render one or more Spark UI pages/tables based on this buffer.

Looked into the logger service, the loggers are executed sequentially, and the loggers are called synchronously.

And checked the LiveListenerBus, it is an asynchronous listener bus for Spark events. SPARK-20863

Yes, this is a spot that we can improve too

Yes, this is a spot that we can improve too

hi @yaooqinn, there seems to be no good way to solve the synchronization problem. For EventLoggerType.SPARK and EventLoggerType.JSON, we need to think out a asynchronous writer for local path or hdfs path.
It seems we should improve it first, because if we listener these events in this pr, it may affects the performance.

Can we start with session Events only in another PR, which can minimize the review burdern

We can combine the history logging and the live data logging  with the listener bus but we can still use the event logging service

Hmm... do not go too far with the spark's listeners and the listener bus if you are not understanding what you are doing here.

I take my early judgment back, and the listener bus looks the right way to go. https://github.com/apache/incubator-kyuubi/pull/1023#discussion_r702801909

Can we start with session Events only in another PR, which can minimize the review burdern

Sure, will do.

there seems to be no good way to solve the synchronization problem.
For EventLoggerType.SPARK

This looks simple, Please send a separate PR to solve this issue
-    sc.eventLogger.foreach(_.onOtherEvent(kyuubiEvent))
+    sc.listenerBus.post(kyuubiEvent)

and EventLoggerType.JSON, we need to think out a asynchronous writer for local path or hdfs

we can implement this latter too

We can combine the history logging and the live data logging with the listener bus but we can still use the event logging service

Had tried serveal times, but failed.
For EventLoggerType.JSON logger, it requires all fields are set.
But for EventLoggerType.SPARK logger, it don't needs all fields.
spark origin event log
{"Event":"SparkListenerJobEnd","Job ID":2,"Completion Time":1630997161352,"Job Result":{"Result":"JobSucceeded"}}
{"Event":"org.apache.spark.sql.execution.ui.SparkListenerSQLExecutionEnd","executionId":3,"time":1630997161355}
{"Event":"org.apache.spark.sql.hive.thriftserver.ui.SparkListenerThriftServerOperationFinish","id":"8f2c5e68-c9fe-4777-bffe-75d171fa18cb","finishTime":1630997161355}
{"Event":"org.apache.spark.sql.hive.thriftserver.ui.SparkListenerThriftServerOperationClosed","id":"8f2c5e68-c9fe-4777-bffe-75d171fa18cb","closeTime":1630997161374}
{"Event":"org.apache.spark.sql.hive.thriftserver.ui.SparkListenerThriftServerSessionClosed","sessionId":"7e3f25d8-5688-4170-8532-5ddee4112991","finishTime":1630997192176}
{"Event":"SparkListenerApplicationEnd","Timestamp":1630997194335}

hi @yaooqinn
Had read the whole ui design of sparkthriftserver
https://github.com/apache/spark/tree/master/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/ui
And already tried toiImplement a memory status store
https://github.com/timothy65535/kyuubi/blob/ky-981-t1/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/ui/EngineStatusStore.scala
At last, in terms of design compatibility and simplicity, I think it's better to reuse the thriftserver's ui design. We need to consider not only the pages on engine server, but also the pages on spark history server.

@yaooqinn already to go, if use customized store, we will convert ElementTrackingStore when support sparkhistoryserver.

can you update the PR description and maybe also some comments in the code changes to help us review?

can you update the PR description and maybe also some comments in the code changes to help us review?

Updated. The store based on TreeBasedStore, it supports row keys and column keys are ordered by natural ordering by default.

cc @cfmcgrady when you are free, thanks.

cc @zhang1002

cc @pan3793 @ulysses-you, help to review when free, thanks.

cc @yaooqinn ready to review, looking for your new advice.

cc @ulysses-you @pan3793

had already tried serval implements:
V1: custom store based on treemap
V2: use spark ElementTrackingStore
V3: custom store based on TreebasedTable, ordered by startTime
V4: custom store based on ConcurrentHashMap, ordered by finishTime
looking for good advice, thanks

BTW, at this stage, seems we don't need to spend a lot of time on details, especially on how to implement the store, it already cost two week.

hi @ulysses-you @pan3793 @cfmcgrady @yaooqinn, any more thought?

cc @ulysses-you @pan3793 @cfmcgrady @yaooqinn, any more thought? thanks

I am +0 on this, but since no one has an opposite option on this implementation, I get this merged.

OK, new idea to the implement of the store is welcome.


Check if player is able to build ( WG and Faction support )

With the last two commits, I added PlotSquared and Factions support. Atm the plugin supports WorldGuard, ASkyblock, PlotSquared and Factions during build permission check.

Thank you! ;)


Add Connection creation with auth_token

  Adding the Connection initiallization with auth_token will allow
  us to use SwiftService not only with: `options['auth'], options['user'],
  options['key'], ...` but with the auth token.

I think this is not needed. You can have the required behavior via os_options:

auth_token
storage_url

Just check out python-swift cli swift auth for the same. I think Connection has everything you need already: https://github.com/openstack/python-swiftclient/blob/eaf40510fad99a3e86f64ecdace46e6e17b89242/swiftclient/client.py#L1764-L1765

Thanks!


python3Packages.labgrid: init at 0.3.1

Motivation for this change
Labgrid is an embedded control and testing library, this package
packages the minimum to let the client and core-library work.
Labgrid also has a check for a specific pyserial port we maintain in the project.
Not sure how to deal with this and whether packaging the fork for nixos is fine.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Labgrid also has a check for a specific pyserial port we maintain in the project.
Not sure how to deal with this and whether packaging the fork for nixos is fine.

If required you can maintain a fork of pyserial in the labgrid project directory.
The gitAndTools.gitstatus package has a similiar relationship to a libgit2 fork if you're interested in how it's done.
https://github.com/NixOS/nixpkgs/blob/eccdb601c1662e96951c1ec8e23791c9d5ef8814/pkgs/applications/version-management/git-and-tools/gitstatus/default.nix#L14
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/version-management/git-and-tools/gitstatus/romkatv_libgit2.nix

Labgrid also has a check for a specific pyserial port we maintain in the project.
Not sure how to deal with this and whether packaging the fork for nixos is fine.

If required you can maintain a fork of pyserial in the labgrid project directory.
The gitAndTools.gitstatus package has a similiar relationship to a libgit2 fork if you're interested in how it's done.
https://github.com/NixOS/nixpkgs/blob/eccdb601c1662e96951c1ec8e23791c9d5ef8814/pkgs/applications/version-management/git-and-tools/gitstatus/default.nix#L14
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/version-management/git-and-tools/gitstatus/romkatv_libgit2.nix

We merged the patches into our pyserial. Upstream should really accept them, and if they don't then eventually downstreams are just going to roll those patches on their own.

If you can tell from my suggestions I can't find anything wrong with this expression.

If you can tell from my suggestions I can't find anything wrong with this expression.

I fixed up both suggestions and added a patch to remove the pyserial check (which uses version and won't work for NixOS any longer, since NixOS applies the patches downstream). Thanks for the review, this should be good to go now.

This looks a bit odd, but it does not seem to cause issues. Do you know whats happening here? @Emantor
=============== 201 passed, 31 skipped, 27 deselected in 36.30s ================
An exception occured during cleanup, call the cleanup() method on targets yourself to handle exceptions explictly.
Error: {} [Errno 2] No such file or directory: 'ssh'
An exception occured during cleanup, call the cleanup() method on targets yourself to handle exceptions explictly.
Error: {} [Errno 2] No such file or directory: 'ssh'
An exception occured during cleanup, call the cleanup() method on targets yourself to handle exceptions explictly.
Error: {} child has vanished
Finished executing pytestCheckPhase

Apart from that I'm also content with the state of this.

This looks a bit odd, but it does not seem to cause issues. Do you know whats happening here? @Emantor
=============== 201 passed, 31 skipped, 27 deselected in 36.30s ================
An exception occured during cleanup, call the cleanup() method on targets yourself to handle exceptions explictly.
Error: {} [Errno 2] No such file or directory: 'ssh'
An exception occured during cleanup, call the cleanup() method on targets yourself to handle exceptions explictly.
Error: {} [Errno 2] No such file or directory: 'ssh'
An exception occured during cleanup, call the cleanup() method on targets yourself to handle exceptions explictly.
Error: {} child has vanished
Finished executing pytestCheckPhase

Apart from that I'm also content with the state of this.

Yes, we do have some tests which setup a mocked SSH connection, but during teardown we try to cleanup all connections. The cleanup tries to use ssh, but this is not available during the test run on NixOS. The cleanup code was added after the tests were introduced and almost every other platform does have ssh available while running the tests. IMO this is fine to ignore for now, I'll fix the tests for the next release so this should vanish on the next update.


"Import Project" screen misstatement

Describe the bug
The "Import Project" screen incorrectly states that OpenRefine project exports can be accepted in .zip and .tar, as per https://groups.google.com/forum/#!topic/openrefine-dev/g5UUWpAQMYM
Expected behavior
Should say "Locate an existing Refine project file (.tar.gz):"
Screenshot

Thanks!

Congrats for finding this!
To solve this, we just need to change the UI text. Changing it in English is enough, as Weblate is able to flag outdated translations to contributors (but in many languages it should not be hard to fix it directly I guess).

I want to work on it

Sure @chetan-v !

Hi @thadguidry I am look on this issue but I cant find the source file to fix this issue

@chetan-v Whenever you see labels (text written) somewhere in OpenRefine, you can do a file content search in the repo for that bit of text written, then it will show which file contains that text.
So do a file search(locally in your IDE or in GitHub) for some of that text: Locate an existing Refine project file

It turns out I was mistaken about this issue:
When you export a project archive to Google Drive, it creates a .zip.
You can download that .zip and import it back into OpenRefine successfully.
So, it accepts .zip and .tar.gz.
I see that #2797 suggests there is a file-type error here; all I know is that OpenRefine successfully interprets its own exported zips as project archives.

Whoops, just saw that #2829 will make this note obsolete. Ignore!

Although named .zip by Google Drive, the contents are actually a .tar.gz. The importer lucks out because the logic tests for != .tar, so anything other than a .tar is assumed to be a .tar.gz. No unzip utility will be able to read those files.


disable CXXOPTS_BUILD_TESTS

update CPM verstion too

Oh thanks for the PR! Didn't see it before creating mine. Closes #94.


loss_classifier: nan (nan)

❓ Questions and Help

Probably too high learning rate. Impossible to say anything else without further information.


Make workflow verbose on failure

Description of the problem

The current CI is verbose for all unit test. This makes it difficult to identify why it crashed

Description of the solution

Fix the ctest command to only output on failure.

I tried the --output-on-failure command on my computer, it can be quite verbose, I'm not sure it helps that much. But I feel it could be relevant when it fails on CI but not on personal computer.

Well the current Verbose would output the same information (which is what is used on the CI, -V)
The only different is that now it would be only verbose when there is an error instead of being always verbose. (You can see if you look at the CI right now for example)


Add basic tests, vndr and travis

This PR adds some basic tests, travis support and vndr for the dependency :)
Feedback please ;-)

In dump.go, what is the purpose of the new print statement? It should probably be up to the library user to decide how errors are displayed (either printed, logged, etc.).
The tests are great. One small thing, near the bottom of the test file there is a variable called expected which contains the mysqldump version (currently 0.1.1). Could you replace the version with the constant version so it doesn't have to be updated.
Finally for vendoring, I personally don't like vendoring directly into the repo and thus use glide for vendoring in my other projects. If you like I can set that up as well as travis.

Hey @JamesStewy ,
sorry, I had some stuff to do ;) Thanks for your feedback. I removed the vendoring, so you're free to set it up like you wish :-)
Regards,
SantoDE

Thanks again @SantoDE :).


sum the same property value for multiple objects

Can I sum the same property value for multiple objects? If I could, how should I write the max expression.

Can I sum the same property value for multiple objects? If I could, how should I write the max expression.

Hello Oneyh,
Could you give an example about what you are trying to do? I couldn't understand what do you mean exactly.

// Just like this
class Subject {
String subjectName;
@Arg("sc")
String subjectScore;

@Arg("tSc")
@Expression("How to write the formula here?")
String totalScore;

public static void main(String[] args) {
    Subject s1 = new Subject();
    s1.subjectName = "math";
    s1.subjectScore = "80";
    Subject s2 = new Subject();
    s2.subjectName = "physics";
    s2.subjectScore = "70";
    Subject s3 = new Subject();
    s3.subjectName = "chemistry";
    s3.subjectScore = "90";

    // I want to assign the total score to the totalScore attributes of each object
}

}

// Just like this
class Subject {
String subjectName;
@Arg("sc")
String subjectScore;

@Arg("tSc")
@Expression("How to write the formula here?")
String totalScore;

public static void main(String[] args) {
    Subject s1 = new Subject();
    s1.subjectName = "math";
    s1.subjectScore = "80";
    Subject s2 = new Subject();
    s2.subjectName = "physics";
    s2.subjectScore = "70";
    Subject s3 = new Subject();
    s3.subjectName = "chemistry";
    s3.subjectScore = "90";

    // I want to assign the total score to the totalScore attributes of each object
}

}

Now I understand, thank you.
There is no way you can do it only with MxReflection. For now, I don't preview to add this feature to the library, Sorry.


Split this mod into API and content

Could you consider splitting this mod into the API which enables the fluid logging and permeability and another mod that acts as the official "Addon" that adds the fluid permeability, fluidloggable, etc... Additionally, adding the ability to use the API to only enable fluidlogging only for specific fluids (whitelisted).
These changes would allow other mod developers (like me) to use your mod as an API to easily add fluidlogging to any custom fluids without creating a dependency that would completely change how the game functions while maintaining a high degree of compatibility with fluid permeability etc.. if the "content" version of this mod is installed.

Great idea
You may have noticed that I haven't worked on this mod for a really long time but your suggestion actually motivated me to continue this mod and it's a good opportunity to actually add comments (I regretted it) to all those mixins.
Thanks a lot 👍
The mod is really messy in my opinion and separating it should help.


TF learn has a problem with discontinuous categorical outputs

I have a train.csv which looks like:
v1,v2,v3
5,4,10
1,2,5
4,5,20
1,11,50
1,12,20

and test.csv:
v1,v2
4,5
2,3
2,4
5,6

The idea is: the output variable (v3) can take four values:  5,10,20,50 and nothing more. It is classification task.
Then code:
import numpy as np
import pandas as pd
from tensorflow.contrib import learn

print ("Reading data")

df_train = pd.read_csv("train.csv")
df_test = pd.read_csv("test.csv")

print ("Dividing to train and test set")

y_train = df_train ['v3']
X_train = df_train [['v1','v2']]

X_test = df_test[['v1','v2']]

classifier_1 = learn.TensorFlowDNNClassifier(
     hidden_units=[20, 10],
     n_classes=4,
     batch_size=128,
     steps=500,
     learning_rate=0.05)

# fit
print ("Fitting")
classifier_1.fit(X_train, y_train)

Then I have a problem:
Traceback (most recent call last):
  File "sample.py", line 26, in <module>
    classifier_1.fit(X_train, y_train)
  File "/Users/qdang/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/estimators/base.py", line 254, in fit
    feed_params_fn=self._data_feeder.get_feed_params)
  File "/Users/qdang/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/trainer.py", line 49, in train
    feed_dict = feed_dict_fn()
  File "/Users/qdang/anaconda/envs/tensorflow/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/io/data_feeder.py", line 281, in _feed_dict_fn
    out.itemset((i, self.y[sample]), 1.0)
IndexError: index 5 is out of bounds for axis 1 with size 4

I think the issue is because the absolute value of output value (5) is larger than the number of classes (4)

@vinhqdang you need to encode your y_train values to [0, 1, 2, 3] classes.
y_train = pd.factorize(df_train['v3'])[0]

Hello,
Afterwards, how could i convert the predicting outputs back to the normal one.
For example, after training a model, and use the model to predict a case, the output is 1.
Of course, 1 is not correct, I need to convert it back to [5,10,20,50] before.

@vinhqdang you'll just save your map somewhere

Hi @terrytangyuan yeah, as discussed in gitter, it is a valid workaround and actually I am using it, but it seems not (very) official way.

@vinhqdang pd.factorize return a tuple; the encoded values and the index/labels.
if you need to know what 1 means you just need to save your labels as @terrytangyuan suggested
y_train, label_index = pd.factorize(df_train['v3'])
# label_index[1] would return the original value encoded to 1

This kind of question should be asked on StackOverflow, since it is a question about how to use TensorFlow rather than a bug in TensorFlow itself.


build docker fail

I try to build the docker,but meet the error
><EMAIL_ADDRESS>build
> next build

sh: 1: next: not found
The command '/bin/sh -c npm run build' returned a non-zero code: 127

Hi @Rainerhu , we're in the process of refactoring the library as the current docker / self-hosting has issues. You can view this PR https://github.com/protectai/rebuff/pull/34 for a working branch / dockerfile in the meantime. I will also update this thread once we've finalized the new self hosting instructions

ok，I  build it sucess use the dockerfile  of the PR。


Installer doesn't work on alpine/musl

If i use the installer script for linux (curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin) on an alpine docker container, i get a glibc build instead of the musl build:
To reproduce, start a docker container (in my case x86_64) with docker run --rm -it alpine and run the following commands in it:
apk add curl file tar
mkdir -p /root/.cargo/bin
curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin
file /root/.cargo/bin/cargo-nextest

Output:
/root/.cargo/bin/cargo-nextest: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.0.0, stripped

Running /root/.cargo/bin/cargo-nextest prints "not found".
Compared with a musl binary:
$ file /bin/tar
/bin/tar: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, BuildID[sha1]=27564d92ae8bd019cb2e3511726a515526a95b5c, stripped

The musl binary from the releases page does work, it seems it's only the installer not checking the platform:
$ file /root/.cargo/bin/cargo-nextest
/root/.cargo/bin/cargo-nextest: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), static-pie linked, stripped

Since the statically linked musl version also works on glibc systems, would it be possible to link the musl version under https://get.nexte.st/latest/linux instead of the glibc version?

Thanks for the report!
Ah ha -- so at the moment we don't actually have an installer script -- that command just untars a binary into the right spot. I think building out an installer script would be interesting. Though I'd want to hook it up to the release metadata we serve in releases.json. This would likely mean that the installer downloads a Rust binary -- which then fetches the release metadata and selects the right version for the platform. This is similar to how rustup-init works.

Going to close this issue out since using the linux-musl version works here.

Since the statically linked musl version also works on glibc systems, would it be possible to link the musl version under https://get.nexte.st/latest/linux instead of the glibc version?

At the moment the musl version has a bug (https://github.com/rust-lang/rust/issues/99740) that the glibc version does not, which is why the glibc version is recommended. However I have work in flight to fix it, likely by the time Rust 1.84 is out. Maybe after that, though in some casual profiling I've seen musl's malloc be significantly slower than glibc's.


Default room config for networks?

There are some configuration options for ROOM that would be nice to set network wide, like maxlines and pastebin support.
Is there a way to do that?
Thanks!

That's something I'd definitely want to do to make all configuration options that affect rooms/channels inheritable and configurable from top level. The current system is very crude and needs a complete redesign to make that work nicely so it hasn't been done yet but I'll keep this open.