Vagrant up fails when using libvirt.uri option.

Vagrant up fails when using libvirt.uri option.
Vagrantfile config
config .vm.define :ctl do |ctl|
ctl.vm.box = "ubuntu-14.04-amd64"
ctl.vm.provision :shell do |s|
s.inline = "sudo bash /vagrant/bootstrap.sh"
end
ctl.vm.provider :libvirt do |libvirt|
ibvirt.uri =<EMAIL_ADDRESS>end
end
Error:
$ vagrant up ctl
Bringing machine 'ctl' up with 'libvirt' provider...
==> ctl: Creating image (snapshot of base box volume).
==> ctl: Creating domain with the following settings...
==> ctl:  -- Name:              stack_ctl
==> ctl:  -- Domain type:       kvm
==> ctl:  -- Cpus:              1
==> ctl:  -- Memory:            512M
==> ctl:  -- Management MAC:
==> ctl:  -- Loader:
==> ctl:  -- Base box:          ubuntu-14.04-amd64
==> ctl:  -- Storage pool:      default
==> ctl:  -- Image:             /var/lib/libvirt/images/stack_ctl.img (40G)
==> ctl:  -- Volume Cache:      default
==> ctl:  -- Kernel:
==> ctl:  -- Initrd:
==> ctl:  -- Graphics Type:     vnc
==> ctl:  -- Graphics Port:     5900
==> ctl:  -- Graphics IP:       <IP_ADDRESS>
==> ctl:  -- Graphics Password: Not defined
==> ctl:  -- Video Type:        cirrus
==> ctl:  -- Video VRAM:        9216
==> ctl:  -- Keymap:            en-us
==> ctl:  -- INPUT:             type=mouse, bus=ps2
==> ctl:  -- Command line :
==> ctl: Creating shared folders metadata...
==> ctl: Starting domain.
==> ctl: Waiting for domain to get an IP address...
==> ctl: Removing domain...
==> ctl: Running cleanup tasks for 'shell' provisioner...
/home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/ssh.rb:47:in initialize': :key_data, :keys, :password or a loaded ssh-agent is required to initialize SSH (ArgumentError)         from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/ssh.rb:9:in new'
from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/ssh.rb:9:in new'         from /home/sdn/.vagrant.d/gems/gems/fog-libvirt-0.0.2/lib/fog/libvirt/models/compute/server.rb:286:in addresses'
from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:39:in block (3 levels) in call'         from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/model.rb:72:in instance_eval'
from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/model.rb:72:in block in wait_for'         from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/wait_for.rb:7:in block in wait_for'
from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/wait_for.rb:6:in loop'         from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/wait_for.rb:6:in wait_for'
from /home/sdn/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/model.rb:69:in wait_for'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:38:in block (2 levels) in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/util/retryable.rb:17:in retryable'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:33:in block in call'
from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/util/timer.rb:9:in time'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/wait_till_up.rb:31:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/start_domain.rb:188:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_boot_order.rb:60:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_network_interfaces.rb:138:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_networks.rb:79:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/share_folders.rb:20:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/prepare_nfs_settings.rb:18:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/synced_folders.rb:86:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/synced_folder_cleanup.rb:28:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/plugins/synced_folders/nfs/action_cleanup.rb:25:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/prepare_nfs_valid_ids.rb:12:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/provision.rb:80:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_domain.rb:201:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/create_domain_volume.rb:51:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_box_image.rb:109:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/handle_box.rb:56:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/handle_storage_pool.rb:50:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /home/sdn/.vagrant.d/gems/gems/vagrant-libvirt-0.0.32/lib/vagrant-libvirt/action/set_name_of_domain.rb:35:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:95:in block in finalize_action'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builder.rb:116:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in block in run'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/util/busy.rb:19:in busy'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in run'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/call.rb:53:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/config_validate.rb:25:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builder.rb:116:in call'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in block in run'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/util/busy.rb:19:in busy'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in run'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:214:in action_raw'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:191:in block in action'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/environment.rb:516:in lock'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:178:in call'
from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:178:in action'         from /opt/vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/batch_action.rb:82:in block (2 levels) in run'
$ vagrant version
Installed Version: 1.7.4
Latest Version: 1.7.4
You're running an up-to-date version of Vagrant!
$vagrant plugin list
vagrant-libvirt (0.0.32)
vagrant-mutate (1.0.3)
vagrant-share (1.1.4, system)
Workaround: Instead of using libvir.uri use other options to connect to libvirt like, libvirt.driver,libvirt.host, libvirt.username, libvirt,password and libvirt.connerct_via_ssh

could u please explain how to use that workaround...i am new on vagrant

I used following options instead of libvirt.uri.
libvirt.host = "<IP_ADDRESS>"
libvirt.username = "admin"
libvirt.password = "password"

thanks :)

Is there any way to get libvirt uri to work ?
I am able to connect libvirt uri over virsh using ssh but when I use vagrant it returns the same error.
/home/username/.vagrant.d/gems/gems/fog-core-1.35.0/lib/fog/core/ssh.rb:47:in `initialize': :key_data, :keys, :password or a loaded ssh-agent is required to initialize SSH (ArgumentError)
here's my provider settings :
config.vm.provider :libvirt do |libvirt|
libvirt.driver = "qemu"
libvirt.connect_via_ssh = true
libvirt.uri =<EMAIL_ADDRESS>libvirt.username = "username"
libvirt.id_ssh_key_file  = "id_dsa"
end
Thanks

The only supported way to use remote libvirt is by using connect_via_ssh - otherwise we cannot at this point determine a VMs IP.
With ssh-agent running and ssh keys in place, I had no problem using the following:
ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'
Vagrant.configure(2) do |config|

  config.vm.define :ctl do |ctl|
      ctl.vm.box = "centos/7"
      ctl.vm.provider :libvirt do |libvirt|
	libvirt.driver = "kvm"
	libvirt.host = "<IP_ADDRESS>"
	libvirt.connect_via_ssh = true
	libvirt.username = "root"
      end
  end
end

Do not specify libvirt.uri together with the rest! As stated in the docs, this overrides all the other config options. You do not want that. Also, if you use ssh-agent, you do not need to specify the key file.


Fix gitignore for Manifest files.

Re-adds Manifest.toml files to the .gitignore.
Fixes the pattern syntax so that Manifests within the AMIP experiments directory are not ignored and can be checked in.
A reminder that in general, Manifests should not be checked in unless we are not using registered versions of dependencies. If this is the case and a Manifest is needed, please add a line to the gitignore with prefix "!" which negates the following pattern rather than deleting Manifest exclusion from gitignore all-together (line 1 of the gitignore). Learn more about gitignore here. You can also always force-add files to git tracking.

bors r+


Author of Chat Manger?

Hello Ivo,
Obey me using the issues feature to get into contact. Winter is coming and i swim in the sea of retro feeling and fired up my old BBS and tried this and that. Also Chat Manger Version 2.52 was a toy of choice. Having some light issues, I asked ChatGPT for a possible Contact to this long discontinued software. And, well, it told me to give GitHub a shot. So i give it a shot to find if you are the genius Author of Chat Manger.
Thank you for reading and the issues can safely be deleted if i wasted your precious time.
Greets,
Cilahn

Wow, Hi Ivo,
nice to meet you in Cyberspace. Back in the day i proudly registered CM for the Twilight Bit BBS 2:244/2370 running under ProBoard 2.16. Currently, i'm toying around to get the old files alive under Linux and dosbox-x. Sadly a lot of files of my Backup nearly 30 years ago got nulled, so progress is slow. But i manged to open up ProBoard to Telnet and trying Multiline Chat in two dosbox-x Windows. I know you warned in the Documentation that chatting to much with yourself is dangerous :-)
The reason i reached out to you is that Chatman works fine, but is ugly slow in Multiline Chats. It takes up 30 seconds updating the opposite Chat Window. I checked the usual suspects as share.exe, nocachedir for Hard drive Access and so on. So i wondered if there is any newer Version than 2.52 which works smoother in Multitasking Environments or utilizes other locking routines. Maybe a hidden command line gem? :-)
Anyway it is about time to say thank you for this great piece of software, which worked great on an i484DX100 and Windows NT 4.0 Workstation.
So is there any hope for a newer Version in your Archives?
Greetings,
Thomas

Thats more nostalgia than I am prone too. Unfortunately I no longer have
the sources and dev environment for chat manager. And you have the latest
version. If i recall correctly it uses files so disabling disk caching as
much as possible might help. Or put those files on a ramdisk or so...
On Mon, 28 Oct 2024, 18:30 cilahn, @.***> wrote:

Wow, Hi Ivo,
nice to meet you in Cyberspace. Back in the day i proudly registered CM
for the Twilight Bit BBS 2:244/2370 running under ProBoard 2.16. Currently,
i'm toying around to get the old files alive under Linux and dosbox-x.
Sadly a lot of files of my Backup nearly 30 years ago got nulled, so
progress is slow. But i manged to open up ProBoard to Telnet and trying
Multiline Chat in two dosbox-x Windows. I know you warned in the
Documentation that chatting to much with yourself is dangerous :-)
The reason i reached out to you is that Chatman works fine, but is ugly
slow in Multiline Chats. It takes up 30 seconds updating the opposite Chat
Window. I checked the usual suspects as share.exe, nocachedir for Hard
drive Access and so on. So i wondered if there is any newer Version than
2.52 which works smoother in Multitasking Environments or utilizes other
locking routines. Maybe a hidden command line gem? :-)
Anyway it is about time to say thank you for this great piece of software,
which worked great on an i484DX100 and Windows NT 4.0 Workstation.
So is there any hope for a newer Version in your Archives?
Greetings,
Thomas
—
Reply to this email directly, view it on GitHub
https://github.com/IvoTops/SQLLike/issues/1#issuecomment-2442206098, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AA5YHTWPUS6SCV37XDYZ4PTZ5ZRBJAVCNFSM6AAAAABQXRVJ2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBSGIYDMMBZHA
.
You are receiving this because you commented.Message ID:
@.***>


Feature: [Back to Top Button]

Feature Request
Is your feature request related to a problem you're trying to solve with Universal-Box? Please describe.

Describe the solution you'd like

How would this feature improve Universal-Box?

Are you willing to resolve this issue by submitting a Pull Request?
Yes

[x ] Yes, I have the time and skills to work on this feature.
[ ] Yes, I have the time, but I would need guidance on how to implement it.
[ ] No, I don't have the time, although I believe I could do it if I had the time...
[ ] No, I don't have the time and I wouldn't even know how to start.

There's no need of such a functionality as of now in our docs website because all the pages are pretty concise


Feature request: include name of journal when dragging journal with image into Conversatin HUD

First: I love this module, and I'm very grateful for your hard work!
I do have a suggestion to try an improve this great tool: when a token is dragged to conversation HUD (either the preparation window, or the actual conversation after it's started), dragging a token automatically includes its name in the label. It would be great if the same thing occurred when I drag a journal with a single image.
Another option (or maybe you could include both) would be able to quickly click or double-click on the label in the conversation HUD to change the name (even if it's empty) on the fly.

I have no idea why I haven't added the image name as well, quite the oversight from my part. I will fix this in the next version of the module.
Regarding the double click to edit, I don't really want to add that (at the moment) as I am not really a fan of double click to edit and I also think it's not very intuitive to the users that they can do that. The feature would also require a bit of rewriting on how the edit functionality works, but maybe I can add something like this in the rework that will be happening sometime in the future.

I need a bit of clarification regarding this issue. I was looking over the code for this, and as it stand, the default behavior when dropping a journal that has an image attached is for the participant that is created to have the name of the image label.

In the example above, when I drop the journal into the conversation, the image caption is used.
Are you using the default FoundryVTT journal image? Can you give me some additional info so I can further look into your issue because on my end it seems fine and I cannot replicate the behavior described.
Thanks in advance!


Improve initial setup time and memory consumption in fast histogram

This is a response to issue #2326.

Modified implementation for approximate quantile sketch for reduced memory consumption
Parallel construction of columnar access structure -- this is a transposed version of input matrix to speed up column-wise access.
Option (use_columnar_access=0) to disable columnar access structure entirely, to further reduce initial setup time and memory usage. Column access may slow down with this option.
Slightly improved feature grouping algorithm

Here are some results on a c3.8xlarge instance (60 GB RAM, 32 cores):

Configuration
setup time
peak memory usage
full log

enable_feature_grouping=1, use_columnar_access=1
190 sec
30.5 GB
log.txt

enable_feature_grouping=0, use_columnar_access=1
69 sec
12.4 GB
log2.txt

enable_feature_grouping=0, use_columnar_access=0
40 sec
12.4 GB
log3.txt

Codecov Report

Merging #2543 into master will decrease coverage by 0.43%.
The diff coverage is 2.8%.

@@            Coverage Diff             @@
##           master    #2543      +/-   ##
==========================================
- Coverage   34.61%   34.18%   -0.44%
==========================================
  Files          76       77       +1
  Lines        7061     7153      +92
  Branches      696      711      +15
==========================================
+ Hits         2444     2445       +1
- Misses       4516     4606      +90
- Partials      101      102       +1

Impacted Files
Coverage Δ

src/common/hist_util.h
0% <ø> (ø)
:arrow_up:

src/tree/updater_histmaker.cc
2.75% <0%> (ø)
:arrow_up:

src/tree/updater_fast_hist.cc
1.08% <0%> (-0.15%)
:arrow_down:

src/common/memory.h
0% <0%> (ø)

src/common/hist_util.cc
0% <0%> (ø)
:arrow_up:

src/common/column_matrix.h
0% <0%> (ø)
:arrow_up:

src/tree/fast_hist_param.h
100% <100%> (ø)
:arrow_up:

src/data/sparse_page_dmatrix.cc
86.88% <0%> (-0.55%)
:arrow_down:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d41dc07...b9da7d4. Read the comment docs.

Is there any reason we can't use a more conventional weighted quantile algorithm (i.e. sort based) here instead of a sketch?

@hcho3 let me know if it is ready to merge

@hcho3 Does this PR changes the training speed other than the setup time? (so I know if I have to redo my benchmarks or not based on this PR)

@hcho3 It crashes on my custom reputation dataset (2,250,000 observations x 23,636 features) when it reaches the grouping feature part. If I disable feature grouping, it works (after 18 minutes of columnar access generation).

@Laurae2 Sorry for the late reply. The improvement was sorely concerned with improving setup
time. Thanks again for your contribution. I do have a few questions for you.

When you say "custom reputation dataset," are you referring to the same dataset you used in issue #2326? I had run tests using the dataset given in #2326 with feature grouping enabled, without any crashes. Let me take another look into this.

Are you still using this machine?

Quad E7-88xx v4 (72 cores), but 3 CPUs were disabled to get rid of NUMA
1TB RAM 1866MHz RAM (256GB RAM available to get rid of NUMA)
Not virtualized
I'm trying to guess whether feature grouping crashes due to lack of memory.

Do you have the full log for the Bosch run (with debug_verbose=1)?

@tqchen I do need to take another look at this pull request. While I'm at it, I'm also inclined to completely re-write the feature grouping logic to make it more parallel. (The logic is inherently sequential now, as it has to inspect one feature at a time.) Let me get back to you on this.

@hcho3 Issue is probably here: https://github.com/hcho3/xgboost/blob/14a33f6cdf2b388f64293b0ac0718d5ab945b37f/src/common/column_matrix.h#L157-L189 (threads are probably sharing a common variable, leading to negative scalability)

It is not exactly the same dataset. My custom reputation dataset can be found using this preprocessing, to do in order:

https://github.com/Laurae2/gbt_benchmarks/blob/master/new_core/creator/reput_setup1.R
https://github.com/Laurae2/gbt_benchmarks/blob/master/new_core/creator/reput_setup2.R
https://github.com/Laurae2/gbt_benchmarks/blob/master/new_core/creator/reput_setup3.R

It leads to the creation of a very large RDS file which contains the dataset (see https://github.com/dmlc/xgboost/issues/2326#issuecomment-313915978 for the training script).

Yes, still using that machine, but with all cores/RAM. I had over 950GB free RAM before it crashed. I noticed the CPU usage was around 5%, xgboost was trying to leverage all cores but it couldn't do so efficiently. Disabling

Unfortunately my server is not available, therefore I tried to reproduce the log on my laptop, It seems I cannot reproduce it on the laptop (I had RAM swap at feature grouping, therefore it is normal it is slow at that part).

However, when it took 8min30 on Bosch I had the same issue as 2., e.g low CPU usage. The very slow part was when xgboost was trying to use all available cores. More exactly, I think it was here: https://github.com/hcho3/xgboost/blob/14a33f6cdf2b388f64293b0ac0718d5ab945b37f/src/common/column_matrix.h#L157-L189
Log for my laptop, i7-4600U:
> model <- xgb.train(params = list(nthread = 4,
+                                  max_depth = 6,
+                                  num_leaves = 63,
+                                  tree_method = "hist",
+                                  grow_policy = "depthwise",
+                                  eta = 0.05,
+                                  max_bin = 255,
+                                  eval_metric = "auc",
+                                  debug_verbose = 2,
+                                  enable_feature_grouping = 1),
+                    data = train,
+                    nrounds = 1,
+                    watchlist = list(test = test),
+                    verbose = 2,
+                    early_stopping_rounds = 50)
[21:05:28] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[21:05:29] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[21:05:33] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 969)...
[21:05:33] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[21:05:37] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[21:05:49] amalgamation/../src/tree/updater_fast_hist.cc:82: Grouping features together...
[21:08:17] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 167.956 sec
[21:08:24] amalgamation/../src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 82 extra nodes, 0 pruned nodes, max_depth=6
[21:08:24] amalgamation/../src/tree/updater_fast_hist.cc:273:
InitData:          0.0625 ( 0.83%)
InitNewNode:       0.0000 ( 0.00%)
BuildHist:         7.2841 (96.83%)
EvaluateSplit:     0.0499 ( 0.66%)
ApplySplit:        0.1260 ( 1.67%)
========================================
Total:             7.5224
[21:08:29] amalgamation/../src/gbm/gbtree.cc:274: CommitModel(): 4.39619 sec
[21:08:29] amalgamation/../src/learner.cc:373: EvalOneIter(): 0.0469153 sec
[1]	test-auc:0.606967
Will train until test_auc hasn't improved in 50 rounds.

[21:08:32] amalgamation/../src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 82 extra nodes, 0 pruned nodes, max_depth=6

@hcho3 parallelizing this part would be useful. I've also noticed that during the initialization lots of time was spent with just a single thread working.

I had accidentally closed the pull request. Sorry for any confusion caused.

@khotilov I was originally planning to postpone the re-write after this pull request, but I changed my mind. Let me go ahead and fix the feature grouping logic.

@Laurae2 Thanks! I will promptly investigate the issue and get back to you.

@hcho3 You can reproduce the issue by doing the following on any machine, even if they do not have 40 threads, on Bosch dataset (or any other large dataset):
model <- xgb.train(params = list(nthread = 40,
                                 max_depth = 6,
                                 num_leaves = 63,
                                 tree_method = "hist",
                                 grow_policy = "depthwise",
                                 eta = 0.05,
                                 max_bin = 255,
                                 eval_metric = "auc",
                                 debug_verbose = 2,
                                 enable_feature_grouping = 1),
                   data = train,
                   nrounds = 2,
                   watchlist = list(test = test),
                   verbose = 2,
                   early_stopping_rounds = 50)

Log of the i7-7700K, which is roughly 2.5x faster (3 min vs 8 min here) than my 72 core server per thread:
[12:52:52] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[12:52:52] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[12:52:55] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 969)...
[12:52:55] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[12:52:56] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[12:56:04] amalgamation/../src/tree/updater_fast_hist.cc:82: Grouping features together...
[12:56:18] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 205.547 sec
[12:56:19] amalgamation/../src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 90 extra nodes, 0 pruned nodes, max_depth=6
[12:56:19] amalgamation/../src/tree/updater_fast_hist.cc:273:
InitData:          0.0029 ( 0.24%)
InitNewNode:       0.0624 ( 5.17%)
BuildHist:         1.1251 (93.29%)
EvaluateSplit:     0.0157 ( 1.30%)
ApplySplit:        0.0000 ( 0.00%)
========================================
Total:             1.2060
[12:56:19] amalgamation/../src/gbm/gbtree.cc:274: CommitModel(): 0.0781817 sec
[12:56:19] amalgamation/../src/learner.cc:373: EvalOneIter(): 0.0336745 sec
[1]	test-auc:0.606966
Will train until test_auc hasn't improved in 50 rounds.

[12:56:20] amalgamation/../src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 86 extra nodes, 0 pruned nodes, max_depth=6
[12:56:20] amalgamation/../src/tree/updater_fast_hist.cc:273:
InitData:          0.0000 ( 0.00%)
InitNewNode:       0.0000 ( 0.00%)
BuildHist:         0.7504 (85.57%)
EvaluateSplit:     0.0953 (10.86%)
ApplySplit:        0.0313 ( 3.57%)
========================================
Total:             0.8770
[12:56:20] amalgamation/../src/gbm/gbtree.cc:274: CommitModel(): 0.0135319 sec
[12:56:20] amalgamation/../src/learner.cc:373: EvalOneIter(): 0.0468612 sec
[2]	test-auc:0.607018

@Laurae2 Sorry for the delay. I've been spending most of my time working on the first release of dmlc/tree-lite. Let me look at it this week for sure. Thanks!

@hcho3 I changed this (https://github.com/Laurae2/ez_xgb/blob/devel/src/common/column_matrix.h#L157) and it's way faster now. Only feature grouping is now causing me issues (crashes).
Old:
[01:47:42] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[01:47:42] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[01:47:44] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 969)...
[01:47:45] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[01:47:45] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[01:55:08] amalgamation/../src/tree/updater_fast_hist.cc:82: Grouping features together...
[01:56:07] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 504.929 sec
[01:56:08] amalgamation/../src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 82 extra nodes, 0 pruned nodes, max_depth=6
[01:56:08] amalgamation/../src/tree/updater_fast_hist.cc:273:
InitData:          0.0156 ( 1.70%)
InitNewNode:       0.0157 ( 1.70%)
BuildHist:         0.7186 (77.95%)
EvaluateSplit:     0.1563 (16.96%)
ApplySplit:        0.0157 ( 1.70%)
========================================
Total:             0.9219
[01:56:08] amalgamation/../src/gbm/gbtree.cc:274: CommitModel(): 0.0940418 sec
[01:56:08] amalgamation/../src/learner.cc:373: EvalOneIter(): 0.0105848 sec
[1]	test-auc:0.606967

New:
[11:10:58] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[11:10:58] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[11:11:00] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 969)...
[11:11:01] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[11:11:01] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[11:11:09] amalgamation/../src/tree/updater_fast_hist.cc:82: Grouping features together...
[11:12:07] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 69.2763 sec
[11:12:08] amalgamation/../src/tree/updater_prune.cc:74: tree pruning end, 1 roots, 82 extra nodes, 0 pruned nodes, max_depth=6
[11:12:08] amalgamation/../src/tree/updater_fast_hist.cc:273:
InitData:          0.0157 ( 1.71%)
InitNewNode:       0.0000 ( 0.00%)
BuildHist:         0.6918 (75.03%)
EvaluateSplit:     0.1832 (19.87%)
ApplySplit:        0.0312 ( 3.39%)
========================================
Total:             0.9220
[11:12:09] amalgamation/../src/gbm/gbtree.cc:205: CommitModel(): 0.0937798 sec
[11:12:09] amalgamation/../src/learner.cc:373: EvalOneIter(): 0.0104179 sec
[1]	test-auc:0.606967

It improved the column access structure generation, and is approx the same speed as LightGBM (clocked also at approximately 4 minutes) when used on my custom reputation dataset:

@Laurae2 With some difficulty, I've managed to reproduce the crash. The crash is really due to run-away memory usage of GHistIndexBlockMatrix. I'm working to write a memory-efficient version of this.
The lines 157-189 of column_matrix.h are not related to the crash. I tried to re-produce negative scaling at this section but could not. With your custom reput dataset, I get 11 sec with 32 threads and 73 sec for 1 thread. What was your value of nthread?

My value of nthread was 40.
I don't have access to my 1TB server, but I have some extra results below.
i7-7700, 1 thread, baremetal server:
[12:39:49] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[12:39:50] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[12:40:29] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 23636)...
[12:41:26] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[12:42:48] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[12:43:51] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 241.215 sec

i7-7700, 8 threads, baremetal server:
[12:31:22] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[12:31:22] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[12:31:29] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 23636)...
[12:32:33] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[12:32:57] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[12:33:28] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 126.432 sec

20 cores Ivy Bridge, 1 thread, virtualized:
[13:05:39] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[13:05:42] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[13:07:19] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 23636)...
[13:08:55] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[13:10:59] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[13:13:09] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 447.5 sec

20 cores Ivy Bridge, 10 threads, virtualized:
[13:27:20] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[13:27:20] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[13:27:28] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 23636)...
[13:29:06] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[13:29:23] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[13:33:30] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 370.016 sec

20 cores Ivy Bridge, 20 threads, virtualized:
[13:14:48] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[13:14:49] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[13:14:57] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 23636)...
[13:16:34] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[13:16:48] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[13:25:36] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 647.326 sec

20 cores Ivy Bridge, 40 threads, virtualized:
[12:43:17] Tree method is selected to be 'hist', which uses a single updater grow_fast_histmaker.
[12:43:18] amalgamation/../src/common/hist_util.cc:37: Generating sketches...
[12:43:25] amalgamation/../src/common/hist_util.cc:75: Computing quantiles for features [0, 23636)...
[12:45:07] amalgamation/../src/tree/updater_fast_hist.cc:70: Quantizing data matrix entries into quantile indices...
[12:45:15] amalgamation/../src/tree/updater_fast_hist.cc:75: Generating columnar access structure...
[13:03:55] amalgamation/../src/tree/updater_fast_hist.cc:92: Done initializing training: 1237.75 sec

With a larger instance, I've re-produce negative scaling:
[11:54:52] src/tree/../common/column_matrix.h:193: nthread = 64, time = 62.0626 sec
[11:55:03] src/tree/../common/column_matrix.h:193: nthread = 32, time = 10.8312 sec
[11:55:12] src/tree/../common/column_matrix.h:193: nthread = 16, time = 8.80852 sec
[11:55:24] src/tree/../common/column_matrix.h:193: nthread = 8, time = 12.4482 sec
[11:55:44] src/tree/../common/column_matrix.h:193: nthread = 4, time = 19.9162 sec
[11:56:23] src/tree/../common/column_matrix.h:193: nthread = 2, time = 39.0732 sec
[11:57:39] src/tree/../common/column_matrix.h:193: nthread = 1, time = 75.285 sec

(I've wrapped the lines 157-189 in a for loop so as to run it with different number of threads.)
But wow, negative scaling is really bad on the Ivy Bridge virtualized machine. I will try to modify this loop to eliminate all data sharing, but if it doesn't work out, I'd be happy with making this sequential.

close due to stale PR, @hcho3

@tqchen Sorry I haven't gotten around working on it for a while. I will send a new PR when it's ready.


fix: Cluster-scoped Rollouts should be restricted to user-defined namepaces

Description
This PR is to modify the logic for cluster scoped rollouts controller. Now cluster scoped rollouts controller is allowed in namespaces that are specified in CLUSTER_SCOPED_ARGO_ROLLOUTS_NAMESPACES env variable.

Addressed review comments, PTAL.


modularization: Dataset Sharing modularization

Modularization of the sharing.
Detail
Migrated the sharing part (including tasks)
Removed unused methods for datasets
There are a few issues that needs to be addressed before merging this PR. But it can be view, since we don't expect major changes here.
Related
#295
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

review remark:
double-checked that stuff purged from frontend/src is indeed unused + if applicable, mutations&resolvers were also removed


Perform direct asset name comparison

Description
Handle situations where the release asset name contains regex symbols that could mess up matching.
Documentation

TODOs
Please ensure all of these TODOs are completed before asking for a review.

[x] Ensure the branch is named correctly with the issue number. e.g: feature/new-vpc-endpoints-955 or bug/missing-count-param-434.
[x] Update the docs.
[x] Keep the changes backward compatible where possible.
[x] Run the pre-commit checks successfully.
[x] Run the relevant tests successfully.
[x] Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.

Related Issues
Fixes #93

I kicked off the build and it passed on CI!

@brikis98 is there a repo where it'll be okay to upload multiple release assets to help with testing? I don't want to pollute any of our actual repos.

It would make sense to use these:
https://github.com/gruntwork-io/fetch-test-public
https://github.com/gruntwork-io/fetch-test-private
Do you have access to these @tonerdo ?

@rhoboat I have access to just https://github.com/gruntwork-io/fetch-test-public but that should suffice

@brikis98 I was only able to add a test for an asset with the + sign. Good thing is that when I try to upload files containing other regex character, GitHub simply converts them to ..


Complete command help information

Before
SUBCOMMANDS:
    create    This is the main structure which stores various commandline options given by high-
              level container runtime
    delete
    events
    exec
    help      Print this message or the help of the given subcommand(s)
    info
    kill
    list      Empty struct for list command
    pause     Structure to implement pause command
    ps        display the processes inside a container
    resume    Structure to implement resume command
    run       Create and start a container. a shortcut for create followed by start
    spec      Command generates a config.json
    start
    state

After
SUBCOMMANDS:
    create    Create a container
    delete    Release any resources held by the container
    events    Show resource statistics for the container
    exec      Execute a process within an existing container
    help      Print this message or the help of the given subcommand(s)
    info      Show information about the system
    kill      Send the specified signal to the container
    list      List created containers
    pause     Suspend the processes within a container
    ps        Display the processes inside a container
    resume    Resume the processes within a container
    run       Create a container and immediately start it
    spec      Create a new runtime specification
    start     Start a previously created container
    state     Show the container state

Codecov Report

Merging #334 (628685a) into main (4224d56) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #334      +/-   ##
==========================================
+ Coverage   71.14%   71.17%   +0.02%
==========================================
  Files          46       46
  Lines        6828     6828
==========================================
+ Hits         4858     4860       +2
+ Misses       1970     1968       -2


Validation error with "Exists" rule

Working Version: Illuminate/Validation: 5.3.4
Broke Version: Illuminate/Validation: 5.3.16 (probably any above 5.3.4 i didn't test all)
PHP Version: PHP 7.0.8-0ubuntu<IP_ADDRESS> (cli) ( NTS )
Database Driver & Version: mysql  Ver 14.14 Distrib 5.7.15, for Linux (x86_64)

Description:
Something happened with this validation... it was working suddenly on the latest version everything broke
i get this error message
Fatal error:  Uncaught Error: Call to undefined method Symfony\Component\Translation\Translator::get() in /var/www/housekeeper/vendor/illuminate/validation/Validator.php:2246
Stack trace:
#0 /var/www/housekeeper/vendor/illuminate/validation/Validator.php(2188): Illuminate\Validation\Validator->getAttribute('room_id')
#1 /var/www/housekeeper/vendor/illuminate/validation/Validator.php(732): Illuminate\Validation\Validator->doReplacements(':attribute \xE4\xB8\x8D\xE5...', 'room_id', 'Exists', Array)
#2 /var/www/housekeeper/vendor/illuminate/validation/Validator.php(715): Illuminate\Validation\Validator->addError('room_id', 'Exists', Array)
#3 /var/www/housekeeper/vendor/illuminate/validation/Validator.php(546): Illuminate\Validation\Validator->addFailure('room_id', 'Exists', Array)
#4 /var/www/housekeeper/vendor/illuminate/validation/Validator.php(461): Illuminate\Validation\Validator->validateAttribute('room_id', 'Exists')
#5 /var/www/housekeeper/vendor/illuminate/validation/Validator.php(486): Illuminate\Validation\Validato in /var/www/housekeeper/vendor/illuminate/validation/Validator.php on line 2246

Steps To Reproduce:
validate something using these rules
$rules = [
  'user_id' => 'required|users,id,deleted_at,NULL'
]

The problem here is with the "deleted_at,NULL" which previously was working...
as laravel will validate even if the user has been trashed... so its important to add deleted_at check on the validation...
The error will occur if you do have a trashed entry on the database
Additional info...
im using the validation out of laravel using this package
"Illuminate/Validation": "^5.3",

and i am using it like
$translator = new \Symfony\Component\Translation\Translator('cn');
$translator->addLoader('json', new \Symfony\Component\Translation\Loader\JsonFileLoader());
$translator->addResource('json', __DIR__.'/validation.json', 'cn');

$validator = new \Illuminate\Validation\Factory(
  $translator, new \Illuminate\Container\Container()
);

$validator->setPresenceVerifier(
  new DatabasePresenceVerifier(.......)
);

$validator->make(.....);

Uhm. Well, the TranslatorInterface doesn't expose a get() method and Symfony's Translator does not have one. Wonder if that code has ever worked.
Anyhow, related PR: https://github.com/laravel/framework/pull/15563

well... it do works.. and i made a repository reproducing the error.. all u need to do is run the code even with a local php on your machine...
then change the composer version from "Illuminate/Validation": "5.3.4" to "Illuminate/Validation": "5.3.16"
and you will see the bug...
https://github.com/Milewski/ValidationBUG

Yes we started using the get() method recently which doesn't exist in Symfony's Translator, I agree this is a breaking change but it was introduced really long time ago not sure if we should revert this change after 1 month.
Anyway we are completely abandoning Symfony's translator in 5.4 which is planned for next month so I really recommend that you start using Laravel's Translator from now.

my project haven't being touch for a month too... then i come to update some things on it.. i faced this issue... its no i have tried to migrate to the new Illuminate/Translation ... but seems things didn't worked with json.. looks i need to make my own implementation on this one.

@themsaid I had the same error and finally arrive to this thread. Can you link Laravel's Translator or explain how to switch it?
thanks

Don't pass the Symfony Translator, use Laravel's translator instead.

@ebisbe now you would go with something like this

use Illuminate\Filesystem\Filesystem;
use Illuminate\Translation\FileLoader;
use Illuminate\Translation\Translator;
use Illuminate\Validation\Factory;
use Illuminate\Container\Container;

$loader = new FileLoader(
  new Filesystem(),'/Translations'
);

$translator = new Translator($loader, 'cn');

$translator->setLocale('cn');
$translator->setFallback('cn');

$validator = new Factory(
 $translator, new Container()
);

the impact it had in my code was that now i need to keep my translations files inside a "laravel structured way" /translations/cn/validation.php which is different from what i was using before... even the file type.. was json.. now php array...

Thanks @Milewski ! I've updated the code an all tests are passing.


training code utilize more cpu memory than gpu memory

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source):
TensorFlow version (1.13.1):
Python version: 2.7
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:10
GPU model and memory:Nvidia RTx 2080

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
I am using tensorflow 1.13.1 with GPU RTx 2080. I am using reinforcement learning to train the robot in a gazebo platform using ROS. The issue is that it is running in CPU memory rather than GPU memory. It takes a lot of time to train a robot. I attached a screenshot of the nvidia smi which shows that it utilize 1.7Gb only of GPU while use 16Gb of RAM.
Describe the expected behavior

Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

@mudassirej Will it be possible us to provide the minimal code snippet which reproduces the reported issue here. Thanks!

@gadagashwini thank you for your consideration. Here is the snippet
from future import print_function
from GazeboWorld import GazeboWorld
import tensorflow as tf
import random
import numpy as np
import time
import rospy
from collections import deque
GAME = 'GazeboWorld'
ACTIONS = 7 # number of valid actions
SPEED = 2 # DoF of speed
GAMMA = 0.99 # decay rate of past observations
OBSERVE = 10. # timesteps to observe before training
EXPLORE = 20000. # frames over which to anneal epsilon
FINAL_EPSILON = 0.0001 # final value of epsilon
INITIAL_EPSILON = 0.1 # starting value of epsilon
REPLAY_MEMORY = 10000# number of previous transitions to remember
BATCH = 1 # size of minibatch
MAX_EPISODE = 20000
MAX_T = 200
DEPTH_IMAGE_WIDTH = 160
DEPTH_IMAGE_HEIGHT = 128
RGB_IMAGE_HEIGHT = 228
RGB_IMAGE_WIDTH = 304
CHANNEL = 3
TAU = 0.001 # Rate to update target network toward primary network
H_SIZE = 81064
IMAGE_HIST = 4
def variable_summaries(var):
"""Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""
with tf.name_scope('summaries'):
mean = tf.reduce_mean(var)
tf.summary.scalar('mean', mean)
with tf.name_scope('stddev'):
stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))
tf.summary.scalar('stddev', stddev)
tf.summary.scalar('max', tf.reduce_max(var))
tf.summary.scalar('min', tf.reduce_min(var))
tf.summary.histogram('histogram', var)
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev = 0.01)
return tf.Variable(initial, name="weights")
def bias_variable(shape):
initial = tf.constant(0., shape = shape)
return tf.Variable(initial, name="bias")
def conv2d(x, W, stride_h, stride_w):
return tf.nn.conv2d(x, W, strides = [1, stride_h, stride_w, 1], padding = "SAME")
class QNetwork(object):
"""docstring for ClassName"""
def init(self, sess):
# network weights
# input 128x160x1
with tf.name_scope("Conv1"):
W_conv1 = weight_variable([10, 14, IMAGE_HIST, 32])
variable_summaries(W_conv1)
b_conv1 = bias_variable([32])
# 16x20x32
with tf.name_scope("Conv2"):
W_conv2 = weight_variable([4, 4, 32, 64])
variable_summaries(W_conv2)
b_conv2 = bias_variable([64])
# 8x10x64
with tf.name_scope("Conv3"):
W_conv3 = weight_variable([3, 3, 64, 64])
variable_summaries(W_conv3)
b_conv3 = bias_variable([64])
# 8x10x64
with tf.name_scope("FCValue"):
W_value = weight_variable([H_SIZE, 512])
variable_summaries(W_value)
b_value = bias_variable([512])
# variable_summaries(b_ob_value)
	with tf.name_scope("FCAdv"):
		W_adv = weight_variable([H_SIZE, 512])
		variable_summaries(W_adv)
		b_adv = bias_variable([512])
		# variable_summaries(b_adv)

	with tf.name_scope("FCValueOut"):
		W_value_out = weight_variable([512, 1])
		variable_summaries(W_value_out)
		b_value_out = bias_variable([1])
		# variable_summaries(b_ob_value_out)

	with tf.name_scope("FCAdvOut"):
		W_adv_out = weight_variable([512, ACTIONS])
		variable_summaries(W_adv_out)
		b_adv_out = bias_variable([ACTIONS])
		# variable_summaries(b_ob_adv_out)

	# input layer
	self.state = tf.placeholder("float", [None, DEPTH_IMAGE_HEIGHT, DEPTH_IMAGE_WIDTH, IMAGE_HIST])
	#print("State:", self.state.shape)
	# Conv1 layer
	h_conv1 = tf.nn.relu(conv2d(self.state, W_conv1, 8, 8) + b_conv1)
	#print("Hidden:", h_conv1)
	# Conv2 layer
	h_conv2 = tf.nn.relu(conv2d(h_conv1, W_conv2, 2, 2) + b_conv2)
	# Conv2 layer
	h_conv3 = tf.nn.relu(conv2d(h_conv2, W_conv3, 1, 1) + b_conv3)
	h_conv3_flat = tf.reshape(h_conv3, [-1, H_SIZE])

	# FC ob value layer
	h_fc_value = tf.nn.relu(tf.matmul(h_conv3_flat, W_value) + b_value)
	value = tf.matmul(h_fc_value, W_value_out) + b_value_out

	# FC ob adv layer
	h_fc_adv = tf.nn.relu(tf.matmul(h_conv3_flat, W_adv) + b_adv)
	advantage = tf.matmul(h_fc_adv, W_adv_out) + b_adv_out

	# Q = value + (adv - advAvg)
	advAvg = tf.expand_dims(tf.reduce_mean(advantage, axis=1), axis=1)
	advIdentifiable = tf.subtract(advantage, advAvg)
	self.readout = tf.add(value, advIdentifiable)
	#print("readout:", self.readout.shape)

	# define the ob cost function
	self.a = tf.placeholder("float", [None, ACTIONS])
	self.y = tf.placeholder("float", [None])
	self.readout_action = tf.reduce_sum(tf.multiply(self.readout, self.a), axis=1)
	self.td_error = tf.square(self.y - self.readout_action)
	self.cost = tf.reduce_mean(self.td_error)
	self.train_step = tf.train.AdamOptimizer(1e-4).minimize(self.cost)

def updateTargetGraph(tfVars,tau):
total_vars = len(tfVars)
op_holder = []
for idx,var in enumerate(tfVars[0:total_vars/2]):
op_holder.append(tfVars[idx+total_vars/2].assign((var.value()*tau) + ((1-tau)*tfVars[idx+total_vars/2].value())))
return op_holder
def updateTarget(op_holder,sess):
for op in op_holder:
sess.run(op)
def trainNetwork():
sess = tf.InteractiveSession()
with tf.name_scope("OnlineNetwork"):
online_net = QNetwork(sess)
with tf.name_scope("TargetNetwork"):
target_net = QNetwork(sess)
rospy.sleep(1.)
reward_var = tf.Variable(0., trainable=False)
reward_epi = tf.summary.scalar('reward', reward_var)
# define summary
merged_summary = tf.summary.merge_all()
summary_writer = tf.summary.FileWriter('./logs1', sess.graph)

# Initialize the World
env = GazeboWorld()
print('Environment initialized')

# Initialize the buffer
D = deque()

# get the first state
#depth_img_t1 = env.GetDepthImageObservation()
#print("input image dimensrion:", depth_img_t1.shape)
#depth_imgs_t1 = np.stack((depth_img_t1, depth_img_t1, depth_img_t1, depth_img_t1), axis=2)
#print("depth images t1:", depth_imgs_t1.shape)
#terminal = False

# saving and loading networks
trainables = tf.trainable_variables()
trainable_saver = tf.train.Saver(trainables)
sess.run(tf.global_variables_initializer())
checkpoint = tf.train.get_checkpoint_state('saved_networks/Q')
print('checkpoint:', checkpoint)
if checkpoint and checkpoint.model_checkpoint_path:
	trainable_saver.restore(sess, checkpoint.model_checkpoint_path)
	print("Successfully loaded:", checkpoint.model_checkpoint_path)

else:
	print("Could not find old network weights")

# start training
episode = 0
epsilon = INITIAL_EPSILON
r_epi = 0.
t = 0
T = 0
rate = rospy.Rate(5)
print('Number of trainable variables:', len(trainables))
targetOps = updateTargetGraph(trainables,TAU)
loop_time = time.time()
last_loop_time = loop_time
while episode < MAX_EPISODE and not rospy.is_shutdown():
	env.ResetWorld()
	t = 0
	r_epi = 0.
	terminal = False
	reset = False
	loop_time_buf = []
	action_index = 0
	while not reset and not rospy.is_shutdown():
		depth_img_t1 = env.GetDepthImageObservation()
		depth_imgs_t1 = np.stack((depth_img_t1, depth_img_t1, depth_img_t1, depth_img_t1), axis=2)
		print("input image dimensrion1:", depth_img_t1.shape)
		depth_img_t1 = np.reshape(depth_img_t1, (DEPTH_IMAGE_HEIGHT, DEPTH_IMAGE_WIDTH, 1))
		print("input image dimensrion after reshape:", depth_img_t1.shape)
		depth_imgs_t1 = np.append(depth_img_t1, depth_imgs_t1[:, :, :(IMAGE_HIST - 1)], axis=2)
		print("input image dimensrion after append1:",depth_imgs_t1.shape)
		reward, terminal, reset = env.GetRewardAndTerminate(t)
		v_t, theta_t = env.GetSelfOdomeSpeed(t)
		if t > 0 :
			D.append((depth_imgs_t, a_t, reward, depth_imgs_t1, terminal))
			if len(D) > REPLAY_MEMORY:
				D.popleft()
		depth_imgs_t = depth_imgs_t1

		# choose an action epsilon greedily
		a = sess.run(online_net.readout, feed_dict = {online_net.state : [depth_imgs_t1]})
		print("action:", a.shape)
		readout_t = a[0]
		a_t = np.zeros([ACTIONS])
		if episode <= OBSERVE:
			action_index = random.randrange(ACTIONS)
			a_t[action_index] = 1
		else:
			if random.random() <= epsilon:
				print("----------Random Action----------")
				action_index = random.randrange(ACTIONS)
				a_t[action_index] = 1
			else:
				action_index = np.argmax(readout_t)
				a_t[action_index] = 1
		# Control the agent
		env.Control(action_index)

		if episode > OBSERVE :
			# # sample a minibatch to train on
			minibatch = random.sample(D, BATCH)
			y_batch = []
			# get the batch variables
			depth_imgs_t_batch = [d[0] for d in minibatch]
			a_batch = [d[1] for d in minibatch]
			r_batch = [d[2] for d in minibatch]
			depth_imgs_t1_batch = [d[3] for d in minibatch]
			Q1 = online_net.readout.eval(feed_dict = {online_net.state : depth_imgs_t1_batch})
			Q2 = target_net.readout.eval(feed_dict = {target_net.state : depth_imgs_t1_batch})
			for i in range(0, len(minibatch)):
				terminal_batch = minibatch[i][4]
				# if terminal, only equals reward
				if terminal_batch:
					y_batch.append(r_batch[i])
				else:
					y_batch.append(r_batch[i] + GAMMA * Q2[i, np.argmax(Q1[i])])

			#Update the network with our target values.
			online_net.train_step.run(feed_dict={online_net.y : y_batch,
												online_net.a : a_batch,
												online_net.state : depth_imgs_t_batch })
			updateTarget(targetOps, sess) # Set the target network to be equal to the primary network.

		r_epi = r_epi + reward
		t += 1
		T += 1
		last_loop_time = loop_time
		loop_time = time.time()
		loop_time_buf.append(loop_time - last_loop_time)
		rate.sleep()

		# scale down epsilon
		if epsilon > FINAL_EPSILON and episode > OBSERVE:
			epsilon -= (INITIAL_EPSILON - FINAL_EPSILON) / EXPLORE

	#  write summaries
	if episode > OBSERVE:
		summary_str = sess.run(merged_summary, feed_dict={reward_var: r_epi})
		summary_writer.add_summary(summary_str, episode - OBSERVE)

	# save progress every 500 episodes
	if (episode+1) % 100 == 0 :
		trainable_saver.save(sess, 'saved_networks/' + GAME + '-dqn', global_step = episode)

	if len(loop_time_buf) == 0:
		print("EPISODE", episode, "/ REWARD", r_epi, "/ steps ", T, v_t, reward_t)
	else:
		print("EPISODE", episode, "/ REWARD", r_epi, "/ steps ", T,
			"/ LoopTime:", np.mean(loop_time_buf),"/ velocity ", v_t, "/ angle ", theta_t, "/ reward current ", reward)

	episode = episode + 1

def main():
trainNetwork()
if name == "main":
main()

Apologies for the delay in response. It is difficult to debug lengthy scripts like these. Perhaps you can provide a minimal code snippet from your whole script which reflects the issue reported. Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

sorry for the late response. I am attaching the snippet of the code and also the status of my gpu and system memory. I need a solution to this problem as I am not able to train my model.


Backticks in topic names break links

As reported here, and further investigated on CZO, it is currently not possible to link to a topic as #stream > topic if the topic name contains backticks.
We should make sure this gets fixed as part of the expected migration to use Djot for markdown processing.

Mm, yeah, closing in favor of https://github.com/zulip/zulip/issues/19873.


"haproxy::globals::sort_options_alphabetic: false" not effective

Describe the Bug
haproxy::globals::sort_options_alphabetic: false is ignored
Expected Behavior
::listen entries not sorted into alphabetical order
Steps to Reproduce

   class { '::haproxy::globals':
     sort_options_alphabetic => false,
   }

trigger puppet run
everything gets ordered alphabetically anyways

Environment

Puppet 7, stdlib9
Debian 10-12

Additional Context
See PR https://github.com/puppetlabs/puppetlabs-haproxy/pull/586

I was just hit with this bug. Is there any chance to push this? @1u1-sdoering can you sign CLA as it seems like that is a showstopper in CI pipeline?

Hi @matejzero , I don't think I can sign anything like that without clearing it with legal. Way too much legalese. But the changes are very trivial, I doubt any CLA is necessary to implement them, with or without my PR..


Allow for sub folders under workflows

Handles the workflows folder recursively, fixing issue #3.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Order REST api setup

I am trying to make a call to place a market order but struggling to initialize the payload for buy market order. below is how it looks currently
`
const data:NewOrder = {
client_order_id: '',
product_id: 'BCH-USD',
side:  OrderSide.BUY,
order_configuration:  ???????
}
cloudClient.rest.order.placeOrder(data).then((resp:any) => {
logger.info("Result of buy is:", resp)
})
`
How do i initialize a market type order?

Please see the doc references I have associated to every method. Here is the official coinbase documentation for placing a market order. The order_configuration of your request should match a schema from there.


Update abomonation_derive to 0.5

We actually have to bump to 0.5 now because the owner of abomonation_derive decided to publish the most recent commit as a minor version upgrade, not a patch level one.

I'm not 100% clear on what is afoot, other than that there are now (pre-PR) lots of errors spewing out from abomonation_derive. Is that what this is about?

Most of the crate eco system moves on to consuming the most recent version of a crate once available (assuming compatibility, of course). abomonation_derive was consuming pre-1.0 versions of proc-macro2, syn, and quote, whereas virtually nobody else was. That results in unnecessary compilation overhead because multiple version of a crate are now being compiled. Presumably it also bloats the binary.
So, by moving to 0.5 we avoid that.
Mind sharing what errors you are referring to?

For example in the cargo doc region of
https://travis-ci.org/TimelyDataflow/timely-dataflow/builds/602816414?utm_source=github_status&utm_medium=notification
I'm not sure the link will work, but if not I can copy paste.
Mostly, I was trying to figure out what the updates to abomonation_derive were and if this update was important for functionality, or just to track things. It's the sort of thing that needs to be coordinated, as downstream users usually need to coordinate too, and I didn't want to do that too randomly.

But, I'm not clear on any reason not to do this, so might as well, right?

Thanks very much for noticing the shift (I had no clue) and for keeping dependencies tidy!

Thanks! For what it's worth, I see the same warnings for Travis run 599933814 which does not include my change.

Oh, I actually meant that without the PR I was seeing the warnings, and I was imagining that perhaps the change was meant to address a related issue.


how does audio work?

really cool project but how do I get audio working?

Ok so basically, install ffmpeg, then, in a command prompt, run:
ffmpeg -y -re -stream_loop -1 -thread_queue_size 4096 -i "https://siloh.pluto.tv/lilo/production/Nick/01/master.m3u8" -vf scale=512:256 -f rawvideo -c:v mjpeg -qscale:v 16 -r 20 udp://<IP_ADDRESS>:1337 -f s16le -acodec pcm_s16le -ac 2 -ar 48000 udp://<IP_ADDRESS>:1338
Then, in game, while looking at the top left corner of where the screen will be, do:
/mcvnc create fard 4 2 false 1000 1337;1338
Now, place the item frames if you haven't already, and put the maps into the item frames (if there were already maps there, either break them via breaking the block behind them, or by temporarily moving [/mcvnc move @] the display [looking at a different block to move it to there])
Now, navigate to port 8819; e.g., http://localhost:8819, and then click connect! The rest of the process explains itself.


Add ability to customize the field for Epic Name

In my JIRA installation, the Epic Name is customfield_10097, so this extension doesn't show them properly for me. I would LOVE to have this feature work!
I think it'd be best if you had an options UI to configure what custom field should be used for epic names.
I'd change the source myself as I know you have it in a config variable but I have no idea how to go about mucking with Chrome extensions. Not sure if you even can do that for extensions installed from the store...

Great idea @OverloadUT. I've wanted to do this, but I wasn't sure if this was something people would  figure out. How did you figure out your Epic Name is customfield_10097? Ideally, I'd like it figure it out automatically, but I'm not sure if this is possible.

I just published a new version that will attempt to auto-discover the right customfield to use. I also fixed some other bugs that might have prevented it from working.
The new version is 2.0.1, Google should approve and start distributing it by Monday morning. Please let me know if it works any better for you.

Excellent! I will be sure to try it out as soon as the new version propagates out.
To answer your question: the way I identified the proper field for my installation was simply by inspecting the DOM for the "Epic Link" field in an issue. Here's an example from mine:
<div id="customfield_10096-val" class="value type-gh-epic-link editable-field inactive" data-fieldtype="gh-epic-link" data-fieldtypecompletekey="com.pyxis.greenhopper.jira:gh-epic-link" title="Click to edit">
So from there I can tell that it's customfield_10096 due to the id of the field.


instance_administrative_notes

create new LDP derived table instance_administrative_notes

Hi @natalyapik! I know the po_instance.sql file stuff is confusing, and hopefully we can get that figured out after your other PR for po_instance.sql is fixed and approved. For now, though, when you actually delete that file from your branch, it is a bit problematic, because then it's like a request to delete that file from the main repository, too.
Instead of deleting the file from your branches, could you just make sure the file on the branches matches the file on the main repo? So, you could download the file from https://github.com/folio-org/folio-analytics/blob/main/sql/derived_tables/po_instance.sql and add it to the sql/derived_tables folder for this branch and the branch for #662. I think that would be the correct fix here, instead of just deleting the file.
Would you be willing to try that while we're waiting for the second reviews to come in?

Hi Angela,
I’m more than willing to do that but I think I need help. Po_instance table got screwed up and also did not make it to 1.4 release. I would like to fix that as well. Can we zoom perhaps and doit together? I am fully open today
Natalya.
From: Angela Zoss @.>
Sent: Friday, August 19, 2022 11:16 AM
To: folio-org/folio-analytics @.>
Cc: Natalya Pikulik @.>; Mention @.>
Subject: Re: [folio-org/folio-analytics] instance_administrative_notes (PR #663)
Hi @natalyapikhttps://github.com/natalyapik! I know the po_instance.sql file stuff is confusing, and hopefully we can get that figured out after your other PR for po_instance.sql is fixed and approved. For now, though, when you actually delete that file from your branch, it is a bit problematic, because then it's like a request to delete that file from the main repository, too.
Instead of deleting the file from your branches, could you just make sure the file on the branches matches the file on the main repo? So, you could download the file from https://github.com/folio-org/folio-analytics/blob/main/sql/derived_tables/po_instance.sql and add it to the sql/derived_tables folder for this branch and the branch for #662https://github.com/folio-org/folio-analytics/pull/662. I think that would be the correct fix here, instead of just deleting the file.
Would you be willing to try that while we're waiting for the second reviews to come in?
—
Reply to this email directly, view it on GitHubhttps://github.com/folio-org/folio-analytics/pull/663#issuecomment-1220796432, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AO5MI7QLF2VGBBEEHZ7I56DVZ6QM3ANCNFSM562EINGA.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Merged manually as https://github.com/folio-org/folio-analytics/commit/f2bb80d712abef4a3d2fbc75e4fb776dbbc916ff


Fixed crash on undefined response

@Alex7Kom this PR should fix the response issue, but untested

I've decided to just check response before the property access: https://github.com/Alex7Kom/node-steam-tradeoffers/commit/89f1e04eae23b959265cfe3477d4cd0973220a0d
Thank you anyway!

:+1:


settings_org: Increase node test coverage.

This increases some coverage for settings_org.

Thanks @lonerz for reviewing, fixed :) let me know if I can improve this.

Merged, thanks @shubhamdhama!


The first paragraph of the documentation for tf.linalg.inv is cut in half

URL(s) with the issue:
Please provide a link to the documentation entry, for example:
https://www.tensorflow.org/api_docs/python/tf/linalg/inv
Description of issue (what needs changing):
The first paragraph says

Computes the inverse of one or more square invertible matrices or their

instead of

Computes the inverse of one or more square invertible matrices or their adjoints (conjugate transposes).

The issue is in:
https://github.com/tensorflow/tensorflow/blob/v2.3.0-rc2/tensorflow/core/api_def/base_api/api_def_MatrixInverse.pbtxt#L18-L22
where the summary spans across the description. Created a PR #41665 for the fix.


Improve the main card and tagline

Hi everyone,
As part of https://github.com/openfoodfacts/smooth-app/issues/5300, we will change a little bit the design of the homepage.
The idea is to keep the carousel but to improve the main card.
Basically, here's a mockup of the change:

There will be a clear hint to scan or search (= this sentence may be hidden on small devices)
The tagline is now in a dedicated card

That's why we have to evolve the content of tagline's JSON to add - at least -:

A title
An image

And to ease the maintenance for @teolemon, a tagline can contain multiple items that will be randomly displayed.
Here's a proposal of the new JSON:
[
   {
      "language":"en",
      "items":[
         {
            "url":"https://translate.openfoodfacts.org/project/openfoodfacts/af",
            "title":"My awesome title",
            "message":"My awesome content",
            "button_label":"Read more",
            "image":{
               "url":"http://world.openfoodfacts.org/logo.png",
               "width":"0.2"
            },
            "start_date":"1717161456000",
            "end_date":"1717161456000",
            "visible":true
         },
         {
            "url":"https://translate.openfoodfacts.org/project/openfoodfacts/af",
            "title":"My awesome title",
            "message":"My awesome content",
            "button_label":"Read more",
            "image":{
               "url":"http://world.openfoodfacts.org/logo.png",
               "width":"0.2"
            },
            "start_date":"1717161456000",
            "end_date":"1717161456000",
            "visible":true
         },
      ]
   }
]

Optional fields:

button_label
start_date and end_date
visible (by default: true)
image (width is in %)

All items will be displayed by their position in the JSON: first item in the first position…
As we now require items, instead of data, I suggest creating separate files and not in /files/tagline-off-ios-v2.json.
If you have any suggestion, feel free to comment 😇

@g123k was mentionned on the server side call yesterday 👍

Here is the new proposal for the JSON:
{
	"news": {
		"nutriscore_v2": {
			"start_date": "08/06/2024 00:00:00",
			"end_date": "31/06/2024 23:59:59",
			"url": "https://openfoodfacts.org/nutriscore-v2",
			"translations": {
				"default": {
					"title": "It's new!",
					"message": "Le **New Calculation** of the NutriScore is available in the application",
					"button_label": "Learn more",
					"image": {
						"url": "https://static.openfoodfacts.org/images/attributes/dist/nutriscore-a-new-en.svg",
						"width": "0.2"
					}
				},
				"fr": {
					"title": "C’est tout nouveau !",
					"message": "Le **Nouveau Calcul** du NutriScore est disponible dans l’application",
					"button_label": "En savoir plus",
					"image": {
						"url": "https://static.openfoodfacts.org/images/attributes/dist/nutriscore-a-new-fr.svg",
						"width": "0.2"
					}
				}
			}
		},
		"demo": {
			"start_date": "",
			"end_date": "",
			"url": "https://translate.openfoodfacts.org/project/openfoodfacts/af",
			"style": {
				"title_background": "#000000",
				"title_text_color": "#FFFFFF",
				"message_background": "#000000",
				"message_text_color": "#FFFFFF",
				"button_background": "#000000",
				"button_text_color": "#FFFFFF"
			},
			"translations": {
				"default": {
					"url": "https://translate.openfoodfacts.org/project/openfoodfacts/en",
					"title": "My title",
					"message": "My message",
					"button_label": "My button",
					"image": {
						"url": "https://static.openfoodfacts.org/images/attributes/dist/nutriscore-a-new-en.svg",
						"width": "0.2"
					}
				},
				"fr": {
					"url": "https://translate.openfoodfacts.org/project/openfoodfacts/fr",
					"title": "C’est tout nouveau !",
					"message": "Le **Nouveau Calcul** du NutriScore est disponible dans l’application",
					"button_label": "En savoir plus",
					"image": {
						"url": "https://static.openfoodfacts.org/images/attributes/dist/nutriscore-a-new-fr.svg",
						"width": "0.2"
					}
				}
			}
		}
	},
	"tagline_feed": {
		"default": {
			"news": [
				{
					"id": "nutriscore_1234"
				}
			]
		},
		"fr_FR": {
			"news": [
				{
					"id": "nutriscore_1234",
					"override": {
						"start_date": "12/06/2024 10:00:00"
					}
				}
			]
		}
	}
}

Now we define a list of news and on the other side, we have the tagline_feed, allowing this JSON to be used for potentially other use cases (e.g.: full-screen message).
For each piece of news, all fields provided in demo are available, including the ability to style it.
Mandatory items are:

id (the key of the item)
url
translations->default

And in a translation:

title
message

In each translation, we can override the url, but that's the only field.
For the tagline_feed, we need to provide at least a default entry with a list of news
For each news, we can override all items from the news.
Last point of attention: to override a date, it needs to be a blank entry and not NULL one.

"In each translation, we can override the url, but that's the only "overridable" field." Why @g123k ?
Is the news key nested in the tagline_feed key mobilizing items from the top level news key ?
Blank is to avoid unintentional overrride, I believe. if you override a date with blank, what happens ? what's the use case ?

"In each translation, we can override the url, but that's the only "overridable" field." Why @g123k ?
The idea is to distinguish the pure content to how and where it will be displayed.
When I say the url is the only field "overridable", it's because start_date and end_date actually are, but on the tagline_feed.

Is the news key nested in the tagline_feed key mobilizing items from the top level news key ?
Yes, the idea is to say, I want this, this and this news. (it doesn't require anymore a visible field)

Blank is to avoid unintentional overrride, I believe. if you override a date with blank, what happens ? what's the use case ?
Eg: You have an end_dateon a news, but in Italy, you don't wan't it.

Updated JSON with:

Image -> alt
Style -> title_indicator_color and content_background_color

An example is available here: https://mock_18ef88079f344225853f73970c3fa2b2.mock.insomnia.rest/files/tagline-off-ios-v3.json


Fully-native version for Android?

I've taken the liberty of throwing together a prototype of a fully native Android version of Private Kit (based on code I've had lying around for a while):
https://github.com/dbrant/privatekit
The rationale? From what I've seen in the past, React Native often fails to account for various OS-specific or even manufacturer-specific quirks, especially when dealing with finicky hardware functionality like location tracking in the background. Eventually you'd need an increasing number of hacks and patches (and chasing after obscure bugs), until they outweigh the effort it would have taken to build a truly native app from the get-go.
Let me know if a fully native version is something you'd like to pursue. I'll be happy to maintain it as a volunteer.

Thank you @dbrant! Right now, we're trying to maximize for rapid development and deployment so staying with a cross-platform framework is necessary at present. Totally hear you on the benefits of native though. If you want to keep maintaining your native fork as we go forward, we may be able to revisit it in the v3 timeframe.
Closing this.


Ubsan

Summary
Minor fix in UBSan.
See discussion in #7285.
Impact
Bug fix.
Testing
I run some test cases in simulator (on 64-bit host), and it yields the correct results.

@fjpanag please rebase your change.


Publicise prototype iPres and Format Aggregator systems for feedback.

After the Goals and Principles blog, we can write follow ups to publicise the Format Aggregator (as is) and the newer iPres index. Compare with other proposals, Ecosystem and NLA/NLNZ. Compare with  https://exponentialdecay.co.uk/blog/pronom-release-statistics/

Refer back to https://anjackson.net/2016/06/08/frontiers-in-format-identification/
Write a summary of the experience, including an initial proposal for improved iPRES metadata, e.g.:

one record for presentation and paper.
make the type clear.

Added some initial notes at: https://github.com/digipres/registries-of-practice-project?tab=readme-ov-file#prototypes and have publicised both on social media.  Still need to write up a blog post before this can be called done.

Okaay, due to deadlines etc., I think I need to write up the notes on aggreging iPRES data first. Sources are here, the digipres/digipres-practice-index README and codebase, and Obsidian.

First part, on aggregation, is live: https://anjackson.net/2024/04/16/aggregating-ipres-proceedings/
Second part to write, about how the prototypes are built. Focus is mostly on user needs, and then maintenance.
Third part needs to lay out some future options.

This was followed by https://anjackson.net/2024/04/29/re-publishing-ipres/
And then formally publicised as https://www.dpconline.org/blog/digital-preservation-publications-index
The Future Options part is less clear and needs writing up separately. Will split into a separate issue.


Class explosion

Right now there is no way to avoid class explosion problem like decorator pattern of gang of four with CDI.
I suggest it should be added like this:
@FacebookDecorator(@SlackDecorator(@SmsDecorator))
Notification notification;

for this example

This issue is extremely vague. It might be just me, but I can't understand what the objection here really is.
For the record, it is impossible to declare annotations in Java so that you can compose them like this: @FacebookDecorator(@SlackDecorator(@SmsDecorator)).

This issue is extremely vague. It might be just me, but I can't understand what the objection here really is.

Same here, more details would be nice :)
BTW, CDI decorator isn't meant to work exactly like the decorator design pattern, if that's the issue. Instead, it is closer to interceptors in its functionality but gives you more power as you have more information about the class you are decorating at the time of invocation.

Google "class explosion" problem once you know that it is pretty clear what the problem is actually.

That doesn't explain much in terms of how is this related to CDI specification?
CDI isn't a design pattern cookbook :shrug:

"CDI isn't a design pattern cookbook" i never said that it was. if you didn't understand the problem pass it to someone who does.
to explain to problem to who didn't get it, lets say we have Notifier class and injected in SmsNotifier, FacebookNotifier and SlackNotifier and lets say we want to combine notifiers right now CDI doesn't have a solution to Class explosion so we have to create SmsAndFacebookNotifier, SmsAndSlackNotifier, FacebookAndSlackNotifier, SmsFacebookAndSlackNotifier classes. 2^n -1 classes needed for each base Notifier class, for 3 base class 7 combinator class, for 5 class 31 combinator class it explodes exponentially hence the name Class explosion.  CDI doesn't provide a way to solve this issue and it is ugly and cumbersome.
That is why there needs to be way to combine injected classes with CDI
@FacebookDecorator(@SlackDecorator(@SmsDecorator))  Notification notification;
if this is not possible then maybe combining with decorator would do like this:
@Decorated  @FacebookDecorator @SlackDecorator @SmsDecorator  Notification notification;

Please stop the passive-aggressive tone.
CDI decorators can actually do what you want with zero class explosion, looking roughly like this:
interface Notifier {
    void notify(Message m);
}

@ApplicationScoped
class DummyNotifier implements Notifier {
    void notify(Message m) {
        // intentionally empty, exists only as an ultimate delegate for decorators
    }
}

@Decorator
@Priority(1)
class FacebookNotifier implements Notifier {
    @Inject
    @Delegate
    Notifier delegate;

    void notify(Message m) {
        sendFacebookNotification(m);
        delegate.notify(m);
    }
}

@Decorator
@Priority(2)
class SlackNotifier implements Notifier {
    @Inject
    @Delegate
    Notifier delegate;

    void notify(Message m) {
        sendSlackNotification(m);
        delegate.notify(m);
    }
}

@Decorator
@Priority(3)
class SmsNotifier implements Notifier {
    @Inject
    @Delegate
    Notifier delegate;

    void notify(Message m) {
        sendSmsNotification(m);
        delegate.notify(m);
    }
}

@ApplicationScoped
class MyService {
    @Inject
    Notifier notifier;

    void doSomething() {
        notifier.notify(new Message("done something"));
    }
}

What might be problematic, depending on the use case, is the declaration-site approach to composition. All injection points of type Notifier get the same "sequence" of decorators.
I assume that what you want is a use-site composition approach, so that each injection point can specify which decorators it wants applied. Do I understand it correctly?
If so, I think something like this might be possible:
@Inject
@DecoratedWith({SmsNotifier.class, SlackNotifier.class})
Notifier noFacebookHere;

Though I didn't really think this through, there might be problems with that that I don't realize right now.
If you have any other ideas, feel free to share, but please, with a complete explanation of the expected behavior. So far, your examples leave a lot for the reader to invent themselves.

Hmm. With such amount of notifiers I would probably rather implement a notifier service with some finer logic. I would expect that sooner or later it would complicate even more.

"I assume that what you want is a use-site composition approach, so that each injection point can specify which decorators it wants applied. Do I understand it correctly?"
Yes that is correct.
"Hmm. With such amount of notifiers I would probably rather implement a notifier service with some finer logic."
it is just a example there might be better examples to address the issue which won't be solved with service.


docker-compose.yml error services.code-server.environment contains an invalid type

there need a - before PASSWORD

We're removing the docker-compose anyway.


Is left-padding in PPO strictly necessary?

I noticed that RemoteExperienceMaker left-pads the input sequences even when using vllm for generation:
https://github.com/OpenLLMAI/OpenRLHF/blob/dcd379a44eea56625626d1a0832cd3eeda048b21/openrlhf/trainer/ppo_utils/experience_maker.py#L346
I can see that a few lines down,self.actor.process_sequences() assumes this left-padding, as it calculates an action mask in a way that hinges on all the inputs terminating at the same index.
Other than that, are there any other parts of the code that assume that inputs are left-padded, respectively that inputs always terminate on the same index?
If not, I'd like to open a PR that skips the left-padding and calculates action mask directly - the left-padding can be inefficient, and action-masking this way doesn't generalise to multi-turn conversations.

Yes this is necessary to reduce the pad of the training samples

Ah, you mean remove_padding_in_sequences()? Wouldn't that still work with only right-padding?

Ah, you mean remove_padding_in_sequences()? Wouldn't that still work with only right-padding?

This will lead to a lot of pads in the middle

Ah, no, to be clear, what I mean is the following:
Right now, the padding is done like this ('promp' - a prompt token, 'respo' - a response token):
| [PAD]   [PAD]   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| [PAD]   [PAD]   [PAD]   promp   promp | respo   respo   respo   [EOS] |

What I have in mind is instead to do this:
| promp   promp   promp | respo   respo   [EOS]   [PAD]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS] |
| promp   promp | respo   respo   respo   [EOS]   [PAD]   [PAD] |

So, less padding overall, and no padding in the middle. The only thing that is now a little different is that the index where the prompt stops and the response starts isn't the same for each sequence - will that break anything?

Ah, no, to be clear, what I mean is the following: Right now, the padding is done like this ('promp' - a prompt token, 'respo' - a response token):
| [PAD]   [PAD]   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| [PAD]   [PAD]   [PAD]   promp   promp | respo   respo   respo   [EOS] |

What I have in mind is instead to do this:
| promp   promp   promp | respo   respo   [EOS]   [PAD]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS] |
| promp   promp | respo   respo   respo   [EOS]   [PAD]   [PAD] |

So, less padding overall, and no padding in the middle. The only thing that is now a little different is that the index where the prompt stops and the response starts isn't the same for each sequence - will that break anything?

they are
| promp   promp   promp  [PAD]  [PAD]      | respo   respo   [EOS]   [PAD]
| promp   promp   promp  promp   promp | respo   respo   [EOS] |  [PAD] |
| promp   promp  [PAD]  [PAD]   [PAD]        | respo   respo   respo   [EOS]

That's not what I am proposing though! What I mean is, if I return it without the pads in the middle from _generate_vllm(), would that break anything? (No worries if I'm not making sense though, I can just try it and see what happens.)

Any updates on this thread? I have the same question. Would it break anything?

Ah, no, to be clear, what I mean is the following: Right now, the padding is done like this ('promp' - a prompt token, 'respo' - a response token):
| [PAD]   [PAD]   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| [PAD]   [PAD]   [PAD]   promp   promp | respo   respo   respo   [EOS] |

What I have in mind is instead to do this:
| prompt   prompt   prompt | response   response   [EOS]   [PAD]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS] |
| promp   promp | respo   respo   respo   [EOS]   [PAD]   [PAD] |

Please provide the source text that you would like translated

So, less padding overall, and no padding in the middle. The only thing that is now a little different is that the index where the prompt stops and the response starts isn't the same for each sequence - will that break anything?

During the training process. Since it is generated in small batches, it needs to be uniformly padded on the left before input, which is determined by the Decoder-Only model.
A completed input and output after generation, since the output in a micro batch is varied in length, padding (PAD) is required for ease of input into the RM for reward calculation. At this point, I believe padding on either the left or right is acceptable, depending on how your RM model's training data is structured.
The PAD method used in the code is
| [PAD]   [PAD]   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| promp   promp   promp   promp   promp | respo   respo   [EOS]   [PAD] |
| [PAD]   [PAD]   [PAD]   promp   promp | respo   respo   respo   [EOS] |

If you want to make full use of these positions, you can rearrange the PAD method in the whole micro batch before inputting the relevant models, such as PM, RM, and Value Model, to save Tokens.


vehicle name must not contain ... spaces: KONA EV

Describe the bug
Since https://github.com/evcc-io/evcc/releases/tag/0.123.1 vehicle name is validated.
It does not allow the vehicle name "KONA EV" now.
I cannot change the name in Bluelink though (just a nickname) and it seems as if EVCC does not detect the car when the name does not match (it is displayed as guest car only when I assign a different name).
Steps to reproduce

Connect a KONA (which is "KONA EV" in Bluelink and not changable)
Add to config
Startup fails with error messsage concerning vehicle name (must not contain special characters or whitespaces)
...

Configuration details
vehicles:
- name: KONA EV
  type: template
  template: hyundai
  title: KONA
  user: ***
  password: ***
  vin: KMHK***
  capacity: 39.2
  language: de
  icon: car
  cache: 5m
  mode: minpv

Log details
Startup fails with error messsage concerning vehicle name (must not contain special characters or whitespaces)

What type of operating system are you running?
HomeAssistant Add-on
Version
0.123.1

Es geht um name- den kannst ändern!


refactor: inject transport service in transports

@jakubkoci as per your comment in #322 here's how you can achieve it using dependency injection
I think we should use injection for the transports in general instead of manually passing the agent in. But for now I think this is a fine solution.

Codecov Report

Merging #368 (376f070) into main (7366ca7) will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #368      +/-   ##
==========================================
+ Coverage   87.81%   87.85%   +0.04%
==========================================
  Files         224      224
  Lines        4127     4125       -2
  Branches      491      491
==========================================
  Hits         3624     3624
+ Misses        503      501       -2

Impacted Files
Coverage Δ

src/agent/Agent.ts
94.11% <ø> (+2.16%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7366ca7...376f070. Read the comment docs.

I still consider transport implementation as something that happens outside of the framework

I see it a something separate from the core of the framework, but I do think it is essential for the ease-of-use of the framework to offer ready-to-use transports. Currently you have to copy the inbound transport, that's just not developer friendly and leaves one of the most complex parts (routing) to the user of the framework.
My idea is to not necessarily have a single HttpInboundTransport, but rather a default ExpressHttpInboundTransport, and later maybe a NestHttpInboundTransport. (either under @aries-framework/node, or as @aries-framework/transport-http-inbound-express)
Can you share the code with me from your experiments of Nest.JS. Maybe I can take a look? If I know where the problem is, I can take this into account for future development. I don't want to exclude a library.


Problem: DHT22 sends minus 0 degree as sign-mag

Problem is related to #52
Solution:
Since all negative temperatures of DHT22 are send in two's complement, it is possible to detect sign-magnitude values and to catch if sensor outputs 0x8000 for a temperature lower then 0°C but higher then -0.1°C

Looks good

uh, I forgott to ingrease version numbers in library.json and library.properties. sorry...

No problem, didn't notice either only after merging :)

0.4.5 released

Instead of setting temperature to 0, it should invert sign and 0x80 should be removed from _bits[2].
This works for me:
int16_t t = ((_bits[2] & 0x7F) * 256 + _bits[3]); if(_bits[2] == 0x80)   _temperature = t * -0.1;  else   _temperature = t * 0.1;


google sheets improvements (wip)

goole sheets stuff moved to typescript and async/await
added deps: promisify, readline-sync
tests

@paulirish thanks. Should we involve Sam? eval(@samccone) 😛

@samccone thanks for the review 👍
I requested review because I don't wanna build just something. I wanna build something good :)
@paulirish @samccone don't hesitate leave comments to my PR. Even if it's a nit picks)
P.S. I'll solve conflicts.

@samccone thanks 🎉


python workspaceFolder bug

Moved from https://github.com/Microsoft/vscode/issues/42787
From @zzylog7

Issue Type
Bug
Description
python debug Error: spawn ${workspaceFolder}\env\Scripts\python.exe ENOENT, "env" is python venv directory , I modify .vscode/setting.json >> python.pythonPath to a full dir and it worked!
VS Code Info
VS Code version: Code - Insiders 1.20.0-insider (519088548b09221e9d590d2e10e14ec6d27ac79a, 2018-02-01T14:21:12.284Z)
OS version: Windows_NT x64 10.0.16299

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz (4 x 2295)

Memory (System)
11.91GB (6.19GB free)

Process Argv
C:\Program Files\Microsoft VS Code Insiders\Code - Insiders.exe

Screen Reader
no

VM
0%

Extensions (14)

Extension
Author (truncated)
Version

yaml
ada
0.0.10

vscode-eslint
dba
1.4.5

tslint
eg2
1.0.24

vscode-great-icons
emm
2.1.24

php-debug
fel
1.12.1

php-intellisense
fel
2.2.5

php-pack
fel
1.0.1

svn-scm
joh
1.13.2

php-cs-fixer
jun
0.1.56

python
ms-
2018.1.0

php-docblocker
nei
1.3.3

autodocstring
njp
0.1.4

vetur
oct
0.11.7

ayu
tea
0.12.0

(1 theme extensions excluded)

Reproduces only with extensions

@zzylog7 please could you go into your work space settings and let me know the exact value of the "python.pythonPath" setting?

This is my workspace settings.json nowadays :
"python.pythonPath": "D:\project\zzy\newAnt\env\Scripts\python.exe"
and before :
"python.pythonPath": "${workspaceFolder}/env/Scripts/python.exe"
I use the python 3.6.4 x64 in my work space and develop in python venv
environment
http://www.baidu.com/link?url=UdvjtP2aTql2_B9-pgfk7mX38GDIk0iTIOZDNcBPZrjvdJw_YqDmR5zX6FDQXctBuohA2S14jg85rcJKINhe3I11-omIl4bmmjyM16vEhRK
.
Hope it is useful to you
2018-02-02 21:46 GMT+08:00 Don Jayamanne<EMAIL_ADDRESS>
@zzylog7 https://github.com/zzylog7 please could you go into your work
space settings and let me know the exact value of the "python.pythonPath"
setting?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vscode-python/issues/691#issuecomment-362590541,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQFJ6CsFWBzhhCRhugfzpNG_200pTiEGks5tQxHSgaJpZM4R21-w
.

@zzylog7 please could you test this on the latest stable version of vscode and the latest version of the extension (not insiders build if vscode).
Please confirm that the folder you have opened in vscode is newAnt.

I'm hitting this too, right after I upgraded to the latest VSCode 1.20.0. I am using a symlink (.virtualenv) that points to a virtualenv. My previous pythonPath was: "${workspaceRoot}/.virtualenv/bin/python". Now, in VSCode 1.20.0, it looks like workspaceRoot support has been completely removed in favor of workspaceFolder. I think this bug is related to that.

I got this bug after I upgraded to vscode 1.20.0, and my pythonPath is "${workspaceFolder}\venv\Scripts\python.exe".
If I change pythonPath to ".\venv\Scripts\python.exe" then everything works fine.

I'm having the same issue as well. vscode 1.20.0; vscode-python 2018.1
This might be related to the multi-root workspace; when I directly open the project, the setting "python.pythonPath": "${workspaceFolder}/.venv/bin/python" works as expected, but when I switch to the multi-root workspace, it does not manage to find the environment (and I can't find it in the prompt Select Interpreter).

Seeing this as well with vscode 1.20.0, vscode-python 2018.1.0. I was not seeing an issue until I updated to vscode 1.20.0 from 1.19.3.
I have my python.pythonPath set to "${workspaceRoot}/.env/Scripts/python.exe" it did not work, changing to "${workspaceFolder}/.env/Scripts/python.exe" also did not work.
As mentioned above changing to a fully qualified path or relative path, ./.env/Scripts/Python.exe, worked. Using the environment selection IO to choose the correct virtualenv also doesn't work since it auto fills it as "${workspaceFolder}/.env/Scripts/python.exe"
No symlinks involved on my end.

@Jasper0819X Thanks for the workaround. I'm new to Python and this issue was making it hard for me to get started.

@YMIndustries @rhummer @ksesong @Jasper0819X @adiun @zzylog7
I do apologize for this issue, unfortunately its an upstream issue in VS Code.
Hopefully we can get this sorted out asap.
@alexandrudima I've created a new issue https://github.com/Microsoft/vscode/issues/43303 for this in the VS Code repo.

@zzylog7 @ksesong, @adiun, @joshsleeper @MetaFight @joshsleeper @adiun @Jasper0819X @rhummer
The fix will be in the next release (end of this month).
Until then please use the Insiders build of this extension as it contains the fix (Note: I'm not referring to VS Code Insiders).
The instructions to download and install this insiders build of the extension can be found here, under Insiders Buid

@DonJayamanne excellent!  Thank you very much :)


Fixing formatting of some paragraphs

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jonyee sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@jonyee Thank you for your contributions. Would you be able to resolve the merge conflicts to proceed with the review?

@jonyee : Nice, good job correcting the whitespace formatting.
Can I ask you to add MarkDown quote indent marker compatibility space (1 blank space) in your modified content?
If yes, are you able to edit and modify the notes yourself, or would you like me to add Github merge suggestions for the lines I am talking about?
Example:
- >[!Important]
+ > [!Important]
- >Note that if you have [Office diagnostic data viewing enabled](#view-office-diagnostic-data), the Office data history is fixed at 1 GB and cannot be modified.
+ > Note that if you have [Office diagnostic data viewing enabled](#view-office-diagnostic-data), the Office data history is fixed at 1 GB and cannot be modified.

@illfated Sure - I added the spaces.  Let me know if that looks right.  The github preview doesn't really look the same as on msdn, so I'm not sure if the markdown is correct, but I think it should be an improvement.

Note to the document author: there is a missing image for the following link:
![Change the size of your data history through the app settings](images/ddv-change-db-size.png)

There is no image file named ddv-change-db-size.png in https://github.com/MicrosoftDocs/windows-itpro-docs/tree/public/windows/privacy/images
If you want that image to be part of the document, it needs to be added to the /windows/privacy/images/ directory.