Add link to File Transfers documentation in helptext for -f, --tar-file-name

Rob Kutschke requested that we add this link to the help text for the -f and the --tar-file-name flags:
https://fifewiki.fnal.gov/wiki/File_Transfers_in_jobsub_lite

So on second thought, I don't want the source code to have any links to fifewiki, since that's meant for FNAL users only.  Rather, I'll duplicate that page on the Github wiki, and then put the github link in the helptext.

I really don't like wiki content being duplicated, unless you have a mechanism to keep them in sync. If you want in on GitHub, fine, but please move it and make the FIFE wiki page a link (or delete it and update the links to it).

I agree with this criticism.  My only hangup is that the Fifewiki page has some information that is FIFE-specific, and I tried to take that out of the github version.
Perhaps I can put a warning at the top of the FIFEWiki page saying that any further updates to that particular page will be made on Github and supply the link?  What do you think?

Maybe we should make the fifewiki one contain ONLY the FIFE-specific details, and have a link to the github page for general info?

I like that idea too.  Let me try that real quick.

How does this look?
https://fifewiki.fnal.gov/wiki/File_Transfers_in_jobsub_lite

That looks good; we should remember though to remove the "What follows here is the ..." and following text later
on so it doesn't confuse people later.

Should I just get rid of that now?
On Tue, Aug 8, 2023, 11:15 AM Marc Mengel @.***> wrote:

That looks good; we should remember though to remove the "What follows
here is the ..." and following text later
on so it doesn't confuse people later.
—
Reply to this email directly, view it on GitHub
https://github.com/fermitools/jobsub_lite/issues/427#issuecomment-1669918562,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEWIQ36EDQKZIDVVBZB2TKLXUJQ2DANCNFSM6AAAAAAYQUEB4E
.
You are receiving this because you were assigned.Message ID:
@.***>

I think yes, go ahead and drop the old text.
--
Marc W. Mengel
Computer Services Senior Developer
Data Management and Applications
Fermi National Accelerator Laboratory
630 840 8256 office
www.fnal.gov
https://email.fnal.gov/owa/redir.aspx?C=JkVsWjSyHEuWV2kFJWNPyC9OyMKf7dEIp_yxPk9yfhfadE3mdLfI6T8Ztf8BVaJjzoIe8oJ5H0M.&URL=http%3A%2F%2Fwww.fnal.gov
Connect with us!
Newsletterhttps://email.fnal.gov/owa/redir.aspx?C=-29AtpIe40SAvGF4aiNr86KvVjap7dEIuxIdg8Q5bYa9sT7QTFVhxhVnl8zwXkEuBSuAQLRkEfg.&URL=http%3A%2F%2Fwww.fnal.gov%2Fpub%2Ftoday | Facebookhttps://email.fnal.gov/owa/UrlBlockedError.aspx | Twitterhttps://email.fnal.gov/owa/redir.aspx?C=-29AtpIe40SAvGF4aiNr86KvVjap7dEIuxIdg8Q5bYa9sT7QTFVhxhVnl8zwXkEuBSuAQLRkEfg.&URL=https%3A%2F%2Ftwitter.com%2FFermilabToday

From: Shreyas Bhat @.>
Sent: Tuesday, August 8, 2023 11:21 AM
To: fermitools/jobsub_lite @.>
Cc: Marc W Mengel @.>; Comment @.>
Subject: Re: [fermitools/jobsub_lite] Add link to File Transfers documentation in helptext for -f, --tar-file-name (Issue #427)
Should I just get rid of that now?
On Tue, Aug 8, 2023, 11:15 AM Marc Mengel @.***> wrote:

That looks good; we should remember though to remove the "What follows
here is the ..." and following text later
on so it doesn't confuse people later.
—
Reply to this email directly, view it on GitHub
https://github.com/fermitools/jobsub_lite/issues/427#issuecomment-1669918562https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_fermitools_jobsub-5Flite_issues_427-23issuecomment-2D1669918562-253E&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=mA4qFs7Hfq-joWhWXnsXfA&m=XB8yNlGv1IVpokdKNeIOdcfRqflPNaoeaLBwbWnfYlSTp6x65qFL3DjbMGfJnMLC&s=MvHihShgZrTfw7767SwVoqzmCv48hfYeAkxXIY9fWeU&e=,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEWIQ36EDQKZIDVVBZB2TKLXUJQ2DANCNFSM6AAAAAAYQUEB4Ehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AEWIQ36EDQKZIDVVBZB2TKLXUJQ2DANCNFSM6AAAAAAYQUEB4E-253E&d=DwQFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=mA4qFs7Hfq-joWhWXnsXfA&m=XB8yNlGv1IVpokdKNeIOdcfRqflPNaoeaLBwbWnfYlSTp6x65qFL3DjbMGfJnMLC&s=LteiIYxonTz0JMiNoGmEpFGKVy4T77QaDfsGjZFV94U&e=
.
You are receiving this because you were assigned.Message ID:
@.***>

—
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_fermitools_jobsub-5Flite_issues_427-23issuecomment-2D1669926629&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=mA4qFs7Hfq-joWhWXnsXfA&m=XB8yNlGv1IVpokdKNeIOdcfRqflPNaoeaLBwbWnfYlSTp6x65qFL3DjbMGfJnMLC&s=eCn-Oktb3mp7W4m0g692unrOqE136TnLAmZg0kTp86g&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAB6GV6PQ67CUPUIZSNT2MDXUJRPHANCNFSM6AAAAAAYQUEB4E&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=mA4qFs7Hfq-joWhWXnsXfA&m=XB8yNlGv1IVpokdKNeIOdcfRqflPNaoeaLBwbWnfYlSTp6x65qFL3DjbMGfJnMLC&s=EjWeKMibx2TrvxPsDU7y9fBp92g2-VFu7H5uHGVaMXI&e=.
You are receiving this because you commented.Message ID: @.***>


Error with MarketIfTouchedOrderRequests for price < 1.0

Hello. In trying out your C# code I found an error in MarketIfTouchedOrderRequests. See file: RestV20Operations.cs, private static async Task Order_RunOrderOperations(). The code works for MarketIfTouchedOrderRequests if price = 1.0 or greater.  However, "USD_CHF" it is currently trading under 1.0. and the code does not work for values less than 1.0.   For example, if you change instrument to "USD_CHF" (looking up the correct instrument values from GetAccountInstrumentsAsync) and change price to 0.945 then the value 0.945 is incorrectly translated into a request in MakeRequestWithJSONBody. The method, requestBody and requestString are correctly passed into MakeRequestWithJSONBody with the correct value of 0.945 but somehow the 0.945 value is changed back from a string into a number somewhere in the code with a value of 0.94499999999999. Suspect it is changed in the StreamWriter. Or is it when a double precision variable is calculated by Trading algorithm is passed to the Math.Round the round function "does not work" (e.g. Visual Studio 2015 IDE displays the rounded value of 0.945 but real value passed to StreamWriter is 0.94499999999999)  See screen capture error below (note I removed userID, accountID and requestID information).

HI mwok,
Thanks for the info.
Many updates and fixes were implemented in the past few weeks.
It's likely your issue was corrected by this commit:  ea1c7fd42bc414a9f5d92e1dd83664c6f5f23436
All EntryOrderRequest subtypes are now serialized via PriceObjectConverter.
PriceObjectConverter correctly rounds the instrument price using the instrument displayPrecision.
So, git pull the latest changes.
Let me know if the issue persists.
/oco

Hi mrchrisok.  Thanks for the quick reply.  I will download the latest version tonight and give it a try.  Note I used your code from early Feb 2018.  But I did not try the one from 11-Feb-2018 yet.  Will try and see if I have the same problem.

Hi mrchirsok. I downloaded your latest code from 12 days ago (about 02-Feb-2018) and there is an error in namespace OkonkwoOandaV20.Framework.Factories   public class ErrorResponseFactory line 14:
switch (dynamicError.type.ToString())
dynamicError.type.ToString() is underlined and the error message is: "A switch expression or case label must be a bool, char, string, integral, enum, or corresponding nullable type

Mwok,
The dev environment for OkonkwoOandaV20 was upgraded to VS2017 and C#7 late last year. C#7 introduced improvements and new capabilities for the switch statement including additional types and pattern matching. A terrific summary article is here (https://www.google.com/url?sa=t&source=web&rct=j&url=http://www.c-sharpcorner.com/article/c-sharp-7-0-and-c-sharp-7-1-new-features-part-two/&ved=2ahUKEwjqntqD_6PZAhWEt1MKHSWSDFgQFjAHegQIBRAB&usg=AOvVaw0PEMmhNVMtIgcIlNABf8eg).
So, a fix for you is to upgrade your environment to VS2017/C#7.  Another is to set a string variable for dynamicError.type.ToString() then use the string variable in the switch.
/oco

Mrchrisok,
Thanks for the tips, help and quick responses.  I will upgrade to VS2017 / C#7, migrate my VS2015 code files over to a fresh build and try it again with the switch not bypassed.
Cheers!
M


Add definitions for power request management

This pull request adds reversed definitions for the following info classes of NtPowerInformation as documented here:

PowerRequestCreate
PlmPowerRequestCreate
PowerRequestAction
GetPowerRequestList

LGTM
I can merge this PR over to PHNT or you can create another over there?

I don't mind if you merge them later the next time you update PHNT with the rest of the changes accumulated here


[SPK] Add support for PRs in introspection tool

As a: developer/manager
I want: to see a column for PRs in the introspection tool
So that: I know which deployments are pending review, who approved past reviews etc.
Describe the solution you'd like:
Refer to this Figma file containing mock ups for how this should look like.

[ ] Add support for PRs to the pipelines and storage table
[ ] Update the models to pull down this information and process it within the Deployment.ts class
[ ] Add this information as column(s) in the spk cli
[ ] Add this information as column(s) in the dashboard

Add support for PRs to the pipelines and storage table

@samiyaakhtar how would this work? Can you describe some high levels steps?

@andrebriggs this was prioritized lower than other items because service creation was still in the process of adding support for PRs. There’s nothing stopping us.

This task has a dependency on https://github.com/microsoft/bedrock/issues/978


Url link is misconfigured on table panel

When clicking on a URL in the table panel it does not open that URL, but it is treated as a relative URL appended to the base path.
Need to fix the following: https://github.com/grafana/grafana-iot-twinmaker-app/blob/3917e1de0187ee72e1585412d2e330cf5c635766/pkg/plugin/twinmaker/utils.go#L133

85528/168655920-78cea18d-0611-4ff4-8856-563ae19bd042.png">

Hm these links seem to correctly redirect for me. Can you or someone on your team double check in case i'm the outlier?


Duplicate glyph names: Disambiguated names are invalid

The addition of hash characters to disambiguate glyph names just bit me again, as the resulting glyph names are invalid.
In my case this is a problem because I'm using an input file in a text format to add information to a font, and there I need to use the glyph names that FontTools is assigning. So I need to use Delta#1, but the input file parser treats the # as a comment marker, thus breaking my input file.
Could we reconsider to change the logic to produce valid glyph names, as Just suggested? Maybe for compilation, we could still support the # notation as not to break existing files.
https://github.com/fonttools/fonttools/blob/1d6dd50d5246e0ee295eb5d2d363c68973be3a43/Lib/fontTools/ttLib/tables/_p_o_s_t.py#L112-L118

the resulting glyph names are invalid

what do you mean by invalid?

the input file parser treats the # as a comment marker

and I suppose you can't change that?

fontTools does not compile invalid postscript names with #, it keeps duplicate glyph names in post table as they were when roundtripping. But since fontTools needs glyph names to be unique it has to internally disambiguate between them by adding the #<digit> suffix.
see my comments in the other thread
https://github.com/fonttools/fonttools/issues/1119#issuecomment-347486107
https://github.com/fonttools/fonttools/issues/1119#issuecomment-347514627

Yes, I have read the comments. I know the names containing a # are not written back into the font. But they are used while the font is decompiled, so every script that deals with the font through TTFont and insists on validity of glyph names will have a problem.
I can change the input format parsing, but it's a bit of a hack, as it requires a certain formatting of comments (I can't change the comment marker that is being used, only under which circumstances it denotes a comment, e.g. only at the start of a line or surrounded by whitespace).
# This is a comment
# The line below does not contain a comment
Delta#1
   # But this again is a comment
  #Whataboutthis?

Doesn't the fact that disambiguation is need point to a problem in the source font?
I'm not opposed to a change here, but I do think that a font containing duplicate glyph names in the post table is problematic in other ways, too.

The source font doesn't actually contain duplicate names in post, but one glyph maps to a standard name (not contained in post) that is also assigned to another glyph via the post table.
The font is generated from FontLab 5 with glyphs Delta (U+0394) and increment (U+2206) FL omits the name for increment from the post table. Below is a font to reproduce.

That's a bit nitpicky... If one glyph refers to the standard mac array and another refers to a "custom" name that happens to be the same as a standard name, then the font still defines two glyphs as having the same name.
If a FontLab file contains a glyph named "increment", yet generates a font that refers to name 168 in the standard list (which is Delta) then what else is that but a bug in FL 5?

By "source font" I didn't mean FL or UFO, just the OTF/TTF that you're operating on, which does contain a duplicate glyph name.

OK, I agree the source font is probably not entirely without issue.
So I will try and fix this in the editor.


Remove default-extensions from .cabal files

They were not needed and it's not good practice.

It seems it was required in one of the test files: NamedFiledPuns changes how shadow warnings work (switches them off). I pushed a commit to fix this.

bors r+


Feature request: don't modify globalConfig.json when running ucc-gen

Description
Thanks @dfederschmidt for this feedback.
meta.version is being modified after an invocation of ucc-gen, which should not be the case.

Only meta.version is being changed after running ucc-gen build which is actually not being used in the UCC UI but this field gives a developer an indication that something changed. I don't mind keeping it how it is.


Update kustomize and go.mod

Update the version of kustomize in Makefile

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@c16f168). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #41   +/-   ##
=======================================
  Coverage        ?   57.21%
=======================================
  Files           ?        4
  Lines           ?      575
  Branches        ?        0
=======================================
  Hits            ?      329
  Misses          ?      222
  Partials        ?       24

Flag
Coverage Δ

unittests
57.21% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


How to do uairctl fetch without format (to use active format)

I'm trying to do uairctl fetch but keep the session format config the same as in the config file, but is that not possible? Because I'd like to have different states with different formats, but fetch it once.

The fetch command was implemented with intention to fetch timer details, like remaining time and mainly timer state(paused or resumed) at any point of time. Can you tell me about your use case? Since, if you need multiple synchronized timers, uairctl listen may cover your needs.

I'm using a gnome extension to put text on the status bar, but the extension only takes in input from programs that actually exit, not from programs that stay running and continuously output.

Implemented with commit be7d23b. Please do note that the implementation depends on the fetch implementation which was somewhat buggy. I'll fix that before the next release.


Don't rewrite log_dir when passing it. [vtcombo]

We were setting the log_dir to vtcombo, however, it was still using the
default value. This should make it only set a default value when
log_dir was not given as input.
Spoke with @sougou about that, and he confirmed that the logs were not going to the folder it was supposed to. This should fix the problem
cc @vmg

Makes perfect sense. 👌

LGTM


Dragging broken in macOS without an eye tracker

The drag command doesn't work for me on Ventura (13.5 or any previous version). I can't select text or move windows or drag files. It just single clicks. Notably, I am not using an eye tracker.
This is not a new issue. I'm still new to Talon and drag has never worked for me (in the public or beta), so I always just assumed it required an eye tracker. Mispronounced something in Google Sheets today and it worked, but I couldn't get it to activate anywhere else. Found issue #974, tried the code, and it now works everywhere.
Per @lunixbochs's suggestion, I've updated things to register the tap on mouse_drag() and unregister things on mouse_drag_end(). Happy to submit a pull request if others can replicate the issue and there's no better way of fixing this.

An event tap in python introduces input lag and stutters, so I don't want the community repo to merge this for everyone. I can likely build a native workaround that doesn't add lag due to taking the GIL.

That would be a much better solution. Thank you!


BUG - Fix Mock Data Labels

Generate data generates multiple data when run. It should check before creating new one if it already exists.


Tarot reversed long

Brick by brick I am adding to the tarot deck, giving each card a long-form description for how to interpret it when it is dealt in the reversed position. Once the deck is amended, I will add the new description to the >tarot card embed.

As a nit, it may be good to do a formatting pass on the YAML once all the descriptions are done. I believe there's.. something in the YAML spec to multi-line strings nicely but I'm not sure what it is - I'll have to do some research in that regard.

As a nit, it may be good to do a formatting pass on the YAML once all the descriptions are done. I believe there's.. something in the YAML spec to multi-line strings nicely but I'm not sure what it is - I'll have to do some research in that regard.

I believe there's.. something in the YAML spec to multi-line strings nicely

https://yaml-multiline.info/

I believe there's.. something in the YAML spec to multi-line strings nicely

https://yaml-multiline.info/

I know I can format the YAML to enforce a char limit, but should I? This YAML is 95% prose and 5% "code," and enforcing a char limit as though the YAML is code would be enforcing a rule for consistency's sake, or so it seems to me. IDEs and the Github website handle word wrap just fine, and the work of formatting the YAML after the fact to a 120-char limit feels pointless.

I know I can format the YAML to enforce a char limit, but should I? This YAML is 95% prose and 5% "code," and enforcing a char limit as though the YAML is code would be enforcing a rule for consistency's sake, or so it seems to me. IDEs and the Github website handle word wrap just fine, and the work of formatting the YAML after the fact to a 120-char limit feels pointless.

I know I can format the YAML to enforce a char limit, but should I? This YAML is 95% prose and 5% "code," and enforcing a char limit as though the YAML is code would be enforcing a rule for consistency's sake, or so it seems to me. IDEs and the Github website handle word wrap just fine, and the work of formatting the YAML after the fact to a 120-char limit feels pointless.

Fair question, and at this point we're bikeshedding, but I'll answer anyway.
Here's a couple reasons why I like splitting it out over multiple lines (usually demarcating around punctuation):

I personally keep word wrap off globally as a consistency thing. I don't typically like word wrap, even in blocks of prose that's inside code. (e.g. a paragraph on an HTML page) Just how my brain works, idk.
If we change the text in the future (fixing typos, etc), since it would be split among multiple lines, the diff would be far easier to read and actually see what's changed.

But I'm not going to block your PR on it. 🤷🏻♂

I know I can format the YAML to enforce a char limit, but should I? This YAML is 95% prose and 5% "code," and enforcing a char limit as though the YAML is code would be enforcing a rule for consistency's sake, or so it seems to me. IDEs and the Github website handle word wrap just fine, and the work of formatting the YAML after the fact to a 120-char limit feels pointless.

Fair question, and at this point we're bikeshedding, but I'll answer anyway.
Here's a couple reasons why I like splitting it out over multiple lines (usually demarcating around punctuation):

I personally keep word wrap off globally as a consistency thing. I don't typically like word wrap, even in blocks of prose that's inside code. (e.g. a paragraph on an HTML page) Just how my brain works, idk.
If we change the text in the future (fixing typos, etc), since it would be split among multiple lines, the diff would be far easier to read and actually see what's changed.

But I'm not going to block your PR on it. 🤷🏻♂

Yoodith wrote their long descriptions for the cards in like three days. I needed three months!


Adding developer-scripts/hpclinux-install.sh

This script has not been tested in its fully-integrated form, but each step was tested manually on HPCLinux.  Something similar should work on most Fedora-based distros, but it's probably better to instead diagnose the problems with the likely cause (stack.sh) on a newer distribution downstream of Fedora. A similar failure was observed on CentOS 6.  I'll investigate this in a virtual machine at some point and create a related issue and fix when I have a better handle on what the specific failure mode is.

:shipit:


Add support for Mac OS X

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

ERROR (python-ace,lib/python3.10/site-packages/pyace/basis.cpython-310-darwin.so): $RPATH/libyaml-cpp-pace.0.6.dylib not found in packages, sysroot(s) nor the missing_dso_whitelist.\n.. is this binary repackaging?', '
ERROR (python-ace,lib/python3.10/site-packages/pyace/evaluator.cpython-310-darwin.so): $RPATH/libyaml-cpp-pace.0.6.dylib not found in packages, sysroot(s) nor the missing_dso_whitelist.\n.. is this binary repackaging?'

Using the patch based on https://github.com/ICAMS/python-ace/pull/30


[evm] add support for Tables

Motivation
This PR implements support for tables in move-to-yul compiler.
Also fixes two small bugs:

When hashing (word, group) in StorageKey function, group should be shifted left to be at the beginning of the second word.
FreezeRef should be treated like an assignment.

/land


Mended grammar for string literals

Added rules to recognise @" and """ strings properly - previously these
broke syntax highlighting.

LGTM thanks

Hi Jared,
I've done the "highlight format strings after
(sprintf|printf[n]|failwithf)" grammar.
It seems to work OK.
It has limitations:

For format strings to be highlighted the printf must be on same line
as the start of the string and separated by only whitespace.
It does not fail to detect some bad format strings - it is pretty
tolerant
It does not explicitly mark bad format strings as bad - if they are
"good enough" they get highlighted otherwise not
highlight colour for format strings is fixed to be same as that for
\ escapes
The code is replicative. There are ways to reuse blocks in the
grammar but I could not be bothered to work them out.

So - not perfect but pretty usable. It does not appear to slow down the
grammar checking, and should not do so due to the way its coded.
Do you want it in this form? The highlight color code be made adjustable
but I'd have to work out how colors work to do this!
Tom
On 29/09/2015 01:56, Jared Hester wrote:

LGTM thanks
—
Reply to this email directly or view it on GitHub
https://github.com/ionide/ionide-fsharp/pull/144#issuecomment-143913719.


[spike]: look into linking with lld

https://nnethercote.github.io/perf-book/compile-times.html suggests we can reduce compile times by linking with lld instead of gcc. a cursory glance indicates its instability is mainly to do with windows, which we do not need to support, however if there are issues even on linux, this is not viable.

From the research, the linux + this linker don't have issues. Just Mac, sometimes Windows
As for speed benefits, 1:45 with or without this option, so not sure if it matters

The link step took super long in an ubuntu VM on my Mac. Will test to see if lld speeds things up

This is the only thing saving me time on the builds, it is saving me 30-40 seconds on the build time, so that is almost a 30% faster build times, but only on subsequent builds

[profile.release]
incremental = true

And with the following got the build time down to 1:10

[build]
rustflags = ["-C", "linker=clang", "-C", "link-arg=-fuse-ld=lld"]

[profile.release]
incremental = true

And with the following got the build time down to 1:10
[build]
rustflags = ["-C", "linker=clang", "-C", "link-arg=-fuse-ld=lld"]

[profile.release]
incremental = true

Tried it with these settings in my VM and there was no noticeable speedup

Closing, no improvements found


Problem moving containers after storage-pools migration

Required information

Distribution: Ubuntu
Distribution version:  16.04LTS and 14.04LTS
The output of "lxc info" or if that fails:

Kernel version: 4.4.0
LXC version: 2.10.1 and 2.10.1
LXD version: 2.10.1 and 2.10.1
Storage backend in use: LVM and dir

Issue description
When attempting to move a container from sys2 (LVM) to sys1 (dir), I get:
root@sys2:~# lxc move test1 sys1:
error: sql: no rows in result set

On the receiving system,  sys1, I see in var/log/lxd/lxd.log:
INFO[03-07|15:40:29] Creating container       ephemeral=false name=test1
INFO[03-07|15:40:29] Deleting container       created=2017-03-07T20:40:29+0000 ephemeral=false name=test1 used=1970-01-01T00:00:00+0000
INFO[03-07|15:40:29] Deleted container        created=2017-03-07T20:40:29+0000 ephemeral=false name=test1 used=1970-01-01T00:00:00+0000

I tried creating a new container test3, and then moved it from sys2 to sys1, started it up, turned it off, and then transferred it back; so new containers seem to be fine.
I confirmed with a couple of other containers as well.
  Prexisting container  => failure
  New container  => success

It appears that lxd.db has not been updated properly. :-(
(Currently looking for differences in lxd.db between new and pre-storage-api containers.)

Can you please also show:

lxc storage volume show container/<container_name>
lxc storage show <pool_name>

Truncated list, but all containers show the same references:
root@sol2:/var/lib/lxd# lxc storage show lxd
config:
  lvm.thinpool_name: LXDPool
  lvm.vg_name: lxd
  source: lxd
  volume.size: 21GB
name: lxd
driver: lvm
used_by:
- /1.0/containers/pcstest
- /1.0/containers/sample-alpine34
- /1.0/containers/sample-jessie
- /1.0/containers/test1
- /1.0/containers/test2
- /1.0/containers/test3
- /1.0/containers/webwork
- /1.0/images/11fc1b1d39b9f9cd7e9491871f1421ac4278e1d599ecf5d180f2a6e2483bd172
- /1.0/images/18e7ed74d0d653894f65343afbc35b92c6781933c273943d882c36a5c5535533
- /1.0/images/457a80ea4720900b69e5542cea5351f58021331bc96e773e4855a3e2ce1e6595
- /1.0/images/543e662b70958f5b87f68b20eb0a205d8c4b14c41f80699e9a98b3b851883d15
- /1.0/images/a570ce23e1dae791e7b8b2f2bcb98c1404273e97c7a1fb972bf0f5835ac3e869
- /1.0/images/d7c16c4fedd3308b5bffdb91f491b8458610c6115d37ace8ba4bcf5c29b23cc6
- /1.0/images/e12c3c1aed259ce62b4a5e8dc5fe8b92d14d36e611b3beae3f55c94df069eeed
- /1.0/profiles/default

root@sys2# lxc storage volume show lxd container/pcstest
name: pcstest
config:
  block.filesystem: ext4
  size: 300GB
type: container
used_by: []

This is strange, because pcstest is actually 10.0GB.
LVM info from lvs:
  containers_pcstest                                                      lxd  Vwi-aotz--  10.00g LXDPool images_543e662b70958f5b87f68b20eb0a205d8c4b14c41f80699e9a98b3b851883d15 8.59 ```
  images_543e662b70958f5b87f68b20eb0a205d8c4b14c41f80699e9a98b3b851883d15 lxd  Vwi-a-tz--  10.00g LXDPool                                                                         8.59

It may be important to know that I attempted to change volume.size to 20GB, but the change was rejected as deprecated. I then edited it back to 300GB and it seemed to accept the value and exit without an obvious error, but now volume.size says 21GB. Something seems messed up.
The target LXD server has 2.3TB in its storage pool; so there is no space constraint at work.

Have to catch my train again. Won't be back on-line for a few hours.

@brauner any idea what's going on with the size property there?
@sbworth Can you paste the output of:

lxc monitor --type=logging     (running in one terminal)
lxc move test1 sys1: --debug   (running in another)

That may give us some hints as to where the database error is coming from.

lxc storage set test volume.size 20GB works fine here even after the upgrade:
<EMAIL_ADDRESS>$%=
> lxc storage set test volume.size 20GB
<EMAIL_ADDRESS>$%=
> lxc storage show test
config:
  lvm.thinpool_name: LXDPool
  lvm.vg_name: test
  source: test
  volume.size: 20GB
name: test
driver: lvm
used_by:
- /1.0/containers/alp1
- /1.0/containers/alp1/snapshots/snap0
- /1.0/containers/alp1/snapshots/snap1
- /1.0/containers/alp1/snapshots/snap2
- /1.0/containers/arch1
- /1.0/containers/arch1/snapshots/snap0
- /1.0/containers/arch1/snapshots/snap1
- /1.0/images/0bbe05d25d3c488ddd712925e8fcb0b514735549a1dfc71a54cc131176468d7e
- /1.0/profiles/default

The "deprecated" warning makes me think that the wrong key may have been set. For example, a daemon configuration key in contrast to the new pool configuration key.

@brauner why does his pool have a "size" property when it's not a loop and why does his container volume have a size which doesn't match the LV?

@stgraber what do you mean by his pool does have a size property? It has a volume.size property which determines the default size to use for new logical volumes. This is retrieved from storage.lvm_volume_size if it was set on the old LXD instance. The size property for the container he is showing also matches what is shown in https://github.com/lxc/lxd/issues/3026 where he seems to have created some logical volumes with 300GB and some with 10GB.

@brauner doh, misread the pool property.

@brauner so why doesn't the size property of his volume match with reality here? Is it just because we take volume.size for all volumes at upgrade time and assume they match reality?

Yeah, if there wasn't anything set on the old LXD instance (a property which we probably didn't have) then volume.size will be assumed as the default.

This will be filled in as the default in storage_volumes_config.go. I can write a patch that parses the size from the logical volume itself.

@brauner yeah, would probably best to set to the current value on upgrade and add an extra patch to fix the values for existing users. If they do manual resizes afterwards then obviously it won't be updated, but that's fine. It'd just be good to have the initial value be accurate at least.

@stgraber, patch will be ready in a minute.

@sbworth, also can you try to restart one of the pre-existing containers and then try to move it again?

I didn't get a chance to do any real work this evening. I'll follow up in the morning.
Restarted a couple of containers. Results look the same:
root@sys2:~# lxc move builder sys1:
error: sql: no rows in result set

Note that I am running lxd 2.10.1, not the modified @stgraber lxd.
More in the morning.

@sbworth, managed to reproduce this locally. Will look into this now.

@stgraber, this is a bug in the root disk device handling code. All of the updated containers have a local root disk device set and LXD doesn't seem to handle this case correctly. Will see what I can come up with.

Ah ha! That sounds like the bug I ran into already in going from 2.8 to 2.9 but was able to finesse around by moving the root disk device definition from the profile into the container (or possibly vice versa as I was not taking notes at the time).
If this is a general issue, you should, of course, fix it in your code base, but if this is something I can work around by moving configuration from container to inherited profile or the reverse, I am happy to to that.
As a side note, this is a lot of code going into lxd for what is supposed to be a transient event: the equivalent of a database migration. Should you perhaps consider simplifying lxd by moving some of that code into a migration utility that can analyze the situation, make necessary changes, and then record API state info somewhere (e.g. a table in lxd.db). You could still choose to have it run by the init system before handing off to lxd, but lxd itself would not need to carry around as much baggage. (Separation of concerns?)

Yeah, you can work around that for now by moving the container from a local root disk device to a profile disk device. Note however, this will only work if you don't have snapshots for that container that also have a root disk device entry. The patches I send will handle all of those cases and will be available with LXD 2.11 which is very likely released today.
It's not just a db migration though since we actually move the container storage either via rsync or if available the fs systems own transfer methods (e.g. btrfs, zfs) and we have that part isolated in a migration module that is separate for each storage backend for aforementioned reasons.

When I mentioned the database, I intended that as an analogy, not literal. You are introducing a major reorganization to your data resources, in this case including both in-database resources and in-filesystem resources, but it is not something you should be doing on an ongoing basis. You transition from stateA -> stateB. lxd_A knows how to deal with a stateA environment, while lxd_B knows how to deal with a stateB environment. You then have a collection of migration operations encapsulated as migrationAB that carries you from stateA to stateB. After achieving stateB, the code in migrationAB is no longer needed. At the moment, you are carrying that code forward in lxd. I worry that carrying the code inside lxd is both a long-term support burden and bloats the binary under normal conditions when that code is unnecessary.
The structure of relational databases makes the problem of state/schema migration more constrained than what you are doing, but in that community it is considered best practice to carve these changes into discrete chunks that are handled as discrete database management procedures, not as normal database operations.
Your code. Your rules.

i have same problem on a dedicated server with lxd installed recently, any fix for this?

LXD 2.11 has the fix


Make it work on Laravel 5.4

I just changed from share() to singleton(), which is supposed to work on any version of Laravel 5.

I don't think merging would make it work on Laravel 5.* since the dependencies are locked to Laravel 4.0.9 till 4.2


Android upload progress

On Android, the progress method never gets called.
RNS3.put(file, options)
  .then(/* ... */)
  .catch(/* ... */)
  .progress(/* never called */);

What version of react native are you using? I know in the past they had not supported upload progress in their XMLHttpRequest abstraction for android, which prompted me to add this guard when trying to add a progress callback. That guard was also present in their examples of using XMLHttpRequest.

Thanks for the quick response! I'm using 0.26.3. Tried to skim the release notes for recent versions, but are they supporting XHR upload progress on Android now?

It looks like they should per their android example in master.
I can try to take a look later but my project is iOS only so I haven't had to use this on android yet. I would expect this library to work on android though, since it's not view related and uses RN's abstractions like XMLHttpRequest which should work the same on iOS and android.
In the meantime, this library is pretty small and it should be straightforward to add some logging to see if your function is being set on this._xhr.upload.onprogress here.

Awesome! Appreciate the feedback and this library you've created!

Ohh another thing I'd try is to run the examples from your current version of RN. Try to run that android example I linked to and see if all is working there. If it is, then you'll know for sure that XMLHttpRequest is working as expected and we'll have to figure out what is going wrong here.

It doesn't work on android, you should update the README to iOS only since it hasn't been tested and currently doesn't work for Android.
I'll work on my own branch and send a PR

@joelwass can you provide any insight into what is not working? This library makes use of standard RN APIs that are supposed to work across iOS and android. So in theory, it should be working for both.
Is the file upload making it to AWS? Is the response handling broken? Nothing working? What version of RN are you on? Is it just the progress method that isn't working, like what this PR was intended for? If not can you open a separate issue with as much detail as you can provide?
Thanks

There is no response, i'm working on error handling because you don't catch an error on the xml request. The xhr has an error but it's not being caught.... from my debugging it's because android has does file URI differently, so a file path in iOS won't work for Android. At least that's my theory, i'm still digging

@joelwass were you able to get this working on Android? If yes, could you please provide your solution?


Make sure hash variable could be changed

The reason behind this PR is that I want to make my URLs look like:
https://foo.com/app/!#some/route
I need to be able to change the hash symbol.
TODO:

[x] Test this PR properly
[x] Add docs

Im done with this PR. See info on the updated readme.

Thank you for merging. Let me know when you tagged 4.3.

@Nyholm there were couple of tests broken after the last merge. I fixed the tests and wrote one more covering the custom hash string. Can you please try 4.3.0 version and let me know if it works properly for you.


[Feature Req] Permission folders (website QoL)

Description
An optional folder feature to help organize permissions in the case where one group is filled with many permissions.
This would be exclusively for the website (/lp editor)
Proposed Behaviour
Adding an optional grouping/folder method for permissions would help a lot with clutter in the case where one of your groups/ranks is filled to the brim with permissions.
This would not affect LP's main behaviour whatsoever, and would just be there as an optional feature to keep users organized in perm management.
(Adding an option to the /lp [User/Group] set command to add that permission in a specific folder would be a plus, but not necessary.)
Extra Details
(Concept Image)

Please consider this! It would make a lot of our lives easier!

Yeah this is something I have been planning to do with the rebuild I'm looking to do :)


Move global uvm_pkg include inside module

Noticed performance impact when running with Cadence xcelium. Debugging showed that global compile units are the reason for that, created by global namespace includes. Cadence strongly recommend to not pollute global namespace.
Checklist:

[ ] tests updated
[ ] documentation updated
[x] CHANGELOG updated

Tested with:

[ ] DSim
[ ] QuestaSim
[ ] VCS
[ ] Verilator
[ ] Vivado
[x] Xcelium

I'm trying to edit the PR to update the tests and the CHANGELOG. The fact that you made it from your fork's master is making it difficult.

The issue has to be open for me to be allowed to push to your fork.

It's going to be difficult to salvage this one, so I made #333 instead. I kept your commit on it.


Add Sen2agri job types to AOI (maja atmos correction, LAI, NDVI, Composites)

Added rudimentary support for basic Sen2agri jobs under the tile list settings panel. Supports L2A, L3A, and L3B only. Needs a lot of work.


⚠ Bofrost.at has degraded performance

In e36efb8, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1656 ms

Resolved: Bofrost.at performance has improved in 89f7ea5 after 9 minutes.