[SPARK-29695][SQL] ALTER TABLE (SerDe properties) should look up catalog/table like v2 commands

What changes were proposed in this pull request?
Add AlterTableSerDePropertiesStatement and make ALTER TABLE ... SET SERDE/SERDEPROPERTIES go through the same catalog/table resolution framework of v2 commands.
Why are the changes needed?
It's important to make all the commands have the same table resolution behavior, to avoid confusing end-users. e.g.
USE my_catalog
DESC t // success and describe the table t from my_catalog
ALTER TABLE t SET SERDE 'org.apache.class' // report table not found as there is no table t in the session catalog

Does this PR introduce any user-facing change?
Yes. When running ALTER TABLE ... SET SERDE/SERDEPROPERTIES, Spark fails the command if the current catalog is set to a v2 catalog, or the table name specified a v2 catalog.
How was this patch tested?
Unit tests.

Test build #113146 has started for PR 26374 at commit b32975f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18060/
Test PASSed.

Test build #113146 has finished for PR 26374 at commit b32975f.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class AlterTableSerDePropertiesStatement(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113146/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18094/
Test PASSed.

Test build #113180 has started for PR 26374 at commit b32975f.

Test build #113180 has finished for PR 26374 at commit b32975f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class AlterTableSerDePropertiesStatement(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113180/
Test PASSed.

@viirya @cloud-fan @imback82
Could you please review? Thanks!

unfortunately it conflicts...

I actually just resolved the conflict a few mins ago before I pinged you. Not sure why it didn't refresh. Let me double check.

Should be good now. Thanks! @cloud-fan

Test build #113219 has started for PR 26374 at commit 00eea57.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18130/
Test PASSed.

Test build #113219 has finished for PR 26374 at commit 00eea57.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class AlterTableSerDePropertiesStatement(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113219/
Test PASSed.

@dongjoon-hyun Thanks for your comments. I will update the code.

Thanks!

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18141/
Test PASSed.

Test build #113231 has started for PR 26374 at commit a38b170.

Test build #113233 has started for PR 26374 at commit 7c2078f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18144/
Test PASSed.

Test build #113231 has finished for PR 26374 at commit a38b170.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113231/
Test PASSed.

Merged to master.

Thanks a lot! @dongjoon-hyun @cloud-fan @viirya @imback82

Test build #113233 has finished for PR 26374 at commit 7c2078f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113233/
Test PASSed.


D-Link switch version bump of external library.

Description: Thanks to @nvella  and @ozrex75 legacy firmware now supports reading power consumption. No changed needed in HA or docs.

Hello @LinuxChristian,
When attempting to inspect the commits of your pull request for CLA signature status among all authors we encountered commit(s) which were not linked to a GitHub account, thus not allowing us to determine their status(es).
The commits that are missing a linked GitHub account are the following:

e39f1edff2167faa67007866edb85e5e4ba4c60d - This commit has something that looks like an email address (cfb@qeye-labs.com). Maybe try linking that to GitHub?.

Unfortunately, we are unable to accept this pull request until this situation is corrected.
Here are your options:

If you had an email address set for the commit that simply wasn't linked to your GitHub account you can link that email now and it will retroactively apply to your commits. The simplest way to do this is to click the link to one of the above commits and look for a blue question mark in a blue circle in the top left. Hovering over that bubble will show you what email address you used. Clicking on that button will take you to your email address settings on GitHub. Just add the email address on that page and you're all set. GitHub has more information about this option in their help center.

If you didn't use an email address at all, it was an invalid email, or it's one you can't link to your GitHub, you will need to change the authorship information of the commit and your global Git settings so this doesn't happen again going forward. GitHub provides some great instructions on how to change your authorship information in their help center.

If you only made a single commit you should be able to rungit commit --amend --author="Author Name<EMAIL_ADDRESS>
(substituting Author Name and<EMAIL_ADDRESS>for your actual information) to set the authorship information.
If you made more than one commit and the commit with the missing authorship information is not the most recent one you have two options:

You can re-create all commits missing authorship information. This is going to be the easiest solution for developers that aren't extremely confident in their Git and command line skills.
You can use this script that GitHub provides to rewrite history. Please note: this should be used only if you are very confident in your abilities and understand its impacts.

Whichever method you choose, I will come by to re-check the pull request once you push the fixes to this branch.

We apologize for this inconvenience, especially since it usually bites new contributors to Home Assistant. We hope you understand the need for us to protect ourselves and the great community we all have built legally. The best thing to come out of this is that you only need to fix this once and it benefits the entire Home Assistant and GitHub community.
Thanks, I look forward to checking this PR again soon! :heart:

Hi @LinuxChristian,
It seems you haven't yet signed a CLA. Please do so here.
Once you do that we will be able to review and accept this pull request.
Thanks!

cherry-picked


Debian packaging: i3ipcpp and xpp

Hi.
I'm starting to pickup my work at packaging polybar for Debian/Ubuntu. The one issue I have at the moment is the embedded libraries in the lib/ directory. I would like to package them separately (i3ipcpp and xpp). I saw that both are forked, and have more code than the original repositories (drmgc/i3ipcpp and jotrk/xpp)
I have a few options:
A) Use your forks (and point to your repos as upstream)
B) Use the original repos, but we wouldn't have the changes needed to make polybar work at 100% likely
C) Use the original repos and carry the diff from your repos as patches (I don't like this suggestion at all)
D) Ask here if it's possible to merge every back to the original repos (I highly doubt this can happen)
What of the above is the one I should pick? I'm open to other suggestions as well.

A is the best option. I don't think B and D would work either, and C just looks like a mess.
Is what you said in #447 still relevant?

Hi.
Yes it's still relevant, but once I package i3ipcpp and xpp (using your forked repos), all should work fine.
I might need a bit of help to make sure I use the installed libraries on the system (instead of the ones shipped in the tarball), but I'll ask when the time comes :)

Hmm, the jsoncpp tarball in the repo is totally useless, I've opened a PR to remove it.
Why do you want to package xpp and i3ipcpp separately? I understand that you can't download anything during the build process, but couldn't you provide the whole polybar repo (including the submodules) to the build servers? I don't know anything about the build process for official debian (or ubuntu for that matter) packages, so what I proposed could not work at all.
Can you give me a quick run through? Are there any constraints on what type of files you can put on the build server or where they come from?
I agree with @NBonaparte that A would work best, if you want to package them separately.
Option D is unfeasible because the turnaround rate for PRs would be way to low. The original xpp is unmaintained and i3ipcpp has barely seen any activity in the last year. Option B probably also won't work, as the original xpp uses Makefiles instead of cmake. As for using upstream i3ipcpp, this one might actually work, however I see that debian has jsoncpp version 1.7.4, in our i3ipcpp fork, we build jsoncpp 1.7.7, if a more recent version is not available, so it is possible that there is some functionality that i3ipcpp uses but that isn't in 1.7.4 (I wasn't able to determine what that would be, but it is possible that jaagr discovered that there is).

@patrick96
I've considered using the embedded copies of i3ipcpp and xpp, however, in the case where another piece of software eventually used any of those, we'd have the same code twice in the repos. In an ideal world, both would be available as a shared library (I might work on that if you're open to it).
The main reason though is security. If there is any vulnerability found, it's easier to patch one library than to patch every program that uses the library. (Obviously this isn't the case with static libraries..)
I'll try to get in touch with the folks maintaining jsoncpp to see if an update is planned soon, thanks for the tip.

Does debian store both the .deb package and the sources it was built from in the repos? Then I understand why you want to avoid duplication.

The main reason though is security. If there is any vulnerability found, it's easier to patch one library than to patch every program that uses the library. (Obviously this isn't the case with static libraries..)

I didn't think about that, but I guess it makes sense that you need to worry about security a lot more than we do in the official repos ;)
However, xpp is a header only library, so any security fix would require the recompilation of all dependent packages anyways. Also i3ipcpp only builds a static library and I don't know how much effort it would be to make it also build a dynamic one that you could then package.
You can check, if polybar compiles with the older jsoncpp version, by changing the following line in the lib/i3ipcpp/CMakeLists.txt:
pkg_check_modules(JSONCPP jsoncpp>=1.7.7)

to
pkg_check_modules(JSONCPP jsoncpp)

Yes both the .deb and source are uploaded. The sources aren't installed on the end-user machines, but are available to anyone who wants them (apt-get source, or via https;//sources.debian.org)
I'll be starting work on those this week and let you know how it goes!
Thanks very much for your time and answers

Glad to help, would be great to have polybar in the official debian repos (and maybe even ubuntu 😉)
If you have anymore questions or problems building, we're here to help

Hi again.
I've started work on i3ipcpp, and so far I have a working package (builds and install the headers and static lib, I will send a PR for that for the 'install' target).
One of the remaining things to do is the copyright file. The "Debian Free Software Guidelines (DFSG)" requires that every file in the source has a license and authors / copyright. Now usually this is easy, but this is the first time working with a fork of a project.
How would you like me to approach this? If I had picked the original repo for example, I would have done:
Files: *
Copyright: Copyright (c) 2015 Sergey Naumov
License: Expat (for MIT)

There were probably different people sending PRs in your own repos, to different files, I'm just unsure how to approach this.

From what I understand about copyright authorship, each contributor has the rights to his or her contributions, so this would theoretically mean that all trivial and non-trivial commits would have to be taken into account.
However, I've looked at the i3 and bspwm Debian packages and they only list the original author despite having numerous contributors.
In that case, I'd probably have drmgc and jaagr for i3ipcpp, and jotrk and jaagr for xpp, so the upstream and the fork creator would have the copyright.
Also, since I'm not too familiar with Debian packaging standards: are all contributors credited or is it usually just the owner, like i3 and bspwm?

It's usually what is contained in the LICENSE file that is shipped. Sometimes though files contain copyright headers with different names, that's when we have to take that into account.
Maybe it could be a good idea to update the License file on both projects to reflect that

I will open two PRs to update the license file with an additional copyright notice, as described in this Stackexchange answer

Just a small update.
I have working packages for both xpp and i3ipcpp. Last time I worked on polybar I wasn't able to get it to work with a "system installed" version of xpp.
Will work a bit more this weekend on it and will ask here if I can't find a solution

Update: I now have packages for i3ipcpp, xpp and polybar that build separately in chroots, and are used together.
I am working with another Debian developer to fix the remaining issues. So far I had to create a patch for polybar to use the system libraries.

That's great to hear.
Just so you know, I'm not sure yet, if we'll want to add the ability for polybar to build against i3ipcpp and xpp packages instead of the submodules to mainline polybar. You may need to add the patches to the source code you upload to debian instead. Because this would just add another way the build process could go wrong when people have mismatched library version or something like this. Also we would now need to keep track of the SONAMEs for the shared libraries and actually have proper release processes for the two submodules. Would this be a problem for your packaging efforts?
Let us know, if we can help you with anything.

Hi.

Would this be a problem for your packaging efforts?

Not really, the patches I made are quite simple (mainly to the CMakeLists.txt files).
I'm still debating if I want to maintain 3 packages over one (with everything included). I had a script made that repacked the tarball with the submodules, but it was quite ugly (was hoping #1120 would be a solution!)
Still waiting on reviews from other Debian folks, but so far so good on my side.
Patches I have so far:
https://salsa.debian.org/jpleau-guest/polybar/blob/split_pkg/debian/patches/use_system_libs.patch
https://salsa.debian.org/jpleau-guest/i3ipcpp/blob/master/debian/patches/adjust_cmake.patch
https://salsa.debian.org/jpleau-guest/i3ipcpp/blob/master/debian/patches/build_shared_lib.patch

A single package would certainly be easier to manage.
As for the script, the following four lines should be everything you need to get a tarball with all the sources.
set -e
git clone --branch 3.1.0 --recursive git://github.com/jaagr/polybar.git
find polybar -name ".git" -exec rm -rf {} \+
tar cf polybar.tar polybar

Hey @jpleau are you still waiting on reviews from the Debian folks?

Hi @patrick96
Yes I am. However even if I was suggested by developers to split up the package, I probably don't have the time to manage all this split whenever a new polybar version comes out, I will definitely try to push for the "one package for everything" approach.

Yes I am.

Alright, just wanted to check in :)

I will definitely try to push for the "one package for everything" approach.

For the sake of your sanity, I am glad you went this route.

I actually haven't had the time to follow up, and to be honest the way it was suggested I package it would put a big burden on me.
I will probably go back to a single package, and upload it to a PPA for the time being..
I'll try to work on that in a not-so-long-future

Eh @jpleau I'm actually learning how to package for Debian and I'll be glad to help in the near future to maintain the package.
Eh btw, are you the jpleau in #debian-quebec on IRC? FYI, my user is sdk.

@dannycolin yes I'm in #debian-quebec :)
For info, the ITP is here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856179

That would be awesome!
Just FYI, for distributed binaries the only optional dependency will be i3, all others (alsa, pulse, mpd, curl, libnl) will become hard dependencies, if polybar is compiled against them. This was discussed in #1406.

Small update, I will have time to pursue this once again, I will first be creating an Ubuntu PPA (For 18.04, and 16.04 if it builds correctly), and move forward for an upload to Debian later on.
Sorry for the delays.

That's cool :D
We're happy for everyone helping out, you really don't have to feel obligated to do this fast or even at all.

@jpleau any update on this issue?

I started clearing my backlog last weekend, polybar should be getting some attention soon. I've hit a few issues in the last couple months with different stuff, I had to put all my debian/ubuntu work on the burner for a bit. Sorry about that.

I'm interested in helping with the Debian package, just pinged @jpleau and the ITP at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=856179

Now we're fairly near to upload the package to the archive \o/

on raspbian I'm getting
dpkg: warning: old libc6:armhf package post-removal script subprocess was killed by signal (Illegal instruc
tion)
dpkg: trying script from the new package instead ...
dpkg: error processing archive /var/cache/apt/archives/libc6_2.30-4_armhf.deb (--unpack):
 new libc6:armhf package post-removal script subprocess was killed by signal (Illegal instruction)
dpkg: error while cleaning up:
 installed libc6:armh
Errors were encountered while processing:
 /var/cache/apt/archives/libc6_2.30-4_armhf.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

dmesg
[   19.009693] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[   19.010217] usbcore: registered new interface driver brcmfmac
[   19.347355] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[   19.347586] brcmfmac: brcmf_c_process_clm_blob: no clm_blob available (err=-2), device may have limited
channels available
[   19.351361] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Oct 22 2019 01:59:28 version 7.45
.98.94 (r723000 CY) FWID 01-3b33decd

@iot-resister What does this have to do with polybar?


Could your support human rating with multiple iterations?

I hope human rating could be embedded in multiple iterations, so that in each iteration I could optimize the prompt toward my goal.

Nice idea !
We'll develop this feature in the near future


feature request: Continue an existing transaction

I've come across a case where a transaction needs to be shared across multiple systems. If we wrap the REST API we can easily achieve this by setting the x-arango-trx-id header. However, we would like to be able receive transaction IDs on both ends and continue the transaction seamlessly using the python-arango interface, instead of crudely performing raw queries against /_cursor.
I've come up with the following hack, which does work, but given that _executor is private, and _executor.id specifically doesn't have a setter, I'm guessing there may be a reason it's discouraged:
from arango.database import TransactionDatabase
from arango.client import ArangoClient

def continue_transaction(db: StandardDatabase, transaction_id):
    trx = TransactionDatabase(connection=deepcopy(db.conn))
    trx._executor._id = transaction_id
    return trx

db = ArangoClient(...).db(...)
trx = continue_transaction(db=db, transaction_id="1234")
trx.collection("vertex").insert({"_key": "test"})
trx.commit_transaction() # Alternatively, don't commit here and let the client who provided the transaction commit it themselves.

Would it make sense to support something like this directly? It seems to me like a reasonable use-case. If so, I'm happy to take a stab at developing a PR for this myself.

Hi @Moortiii,
I understand your proposal, and I think it is quite sensible. Updating the same transaction concurrently can some uncertainty due to timing issues, but when carefully, I can imagine some valid use-cases.
As you pointed out, the _executor.id is indeed private. While adding a setter would be the easy way out of this, it would potentially allow users to write code like this:
trx = db.begin_transaction()
col1 = trx.collection("col1")
trx._executor._id = another_transaction
col2 = trx.collection("col2")

Not only the transaction can easily get lost, thus preventing one from ever accessing the initial transaction again, but the problem can be easily overlooked, as it is hidden in just one line of code. Frankly, I believe even the x-arango-trx-id setting trick is way better - it may look weird, but it's "loud and clear", there will be no problem figuring out what (and why) you wrote it there.
Following up on what I would consider a reasonable solution

Modify the TransactionAPIExecutor constructor such that it contains a new field, transaction_id, which can be None or str (basically an Optional[str]). In case it is a str, the constructor should no longer send a request to /_api/transaction/begin, but set the _id property directly and check the status() of the transaction in order to validate it really exists.
The same parameter should be added to the TransactionDatabase, which would forward it to the executor. This is straight forward.
The StandardDatabase should get a fetch_transaction method, which takes the transaction ID and returns a TransactionDatabase. I'm suggesting fetch_transaction because it implies that a transaction may (or may not) be there, rather than continuing one (which is not necessarily "paused").

Testing
Introduce a test case test_transaction.py, something simple, just to check that we're able to use both the initial transaction and the "continued" object.
def test_transaction_fetch(db, col, docs):
    txn_db = db.begin_transaction(write=col.name)
    txn_col = txn_db.collection(col.name)
    txn_db2 = db.fetch_transaction(txn_db.transaction_id)
    # insert some documents using both txn's
    # ...

Docs
A small edit in _transaction.rst` would be great to showcase how fetch_transaction is supposed to be used.
I'm ready to implement the above. Or, if you want to give it a go, I'm perfectly fine with that, but don't feel pressured, I'm just mentioning since you offered.
Let me know what you think.

These seem like sensible changes that should be straight forward enough to implement. I'll give it a shot later today and report back. Thanks!I agree with your comment about continue which could imply the ability to "pause" a transaction. I hadn't thought about it that way, but you're probably right that it would cause some confusion, especially for new users of ArangoDB.On Mar 10, 2024, at 16:24, Alex Petenchea @.***> wrote:
Hi @Moortiii,
I understand your proposal, and I think it is quite sensible. Updating the same transaction concurrently can some uncertainty due to timing issues, but when carefully, I can imagine some valid use-cases.
As you pointed out, the _executor.id is indeed private. While adding a setter would be the easy way out of this, it would potentially allow users to write code like this:
trx = db.begin_transaction()
col1 = trx.collection("col1")
trx._executor._id = another_transaction
col2 = trx.collection("col2")
Not only the transaction can easily get lost, thus preventing one from ever accessing the initial transaction again, but the problem can be easily overlooked, as it is hidden in just one line of code. Frankly, I believe even the x-arango-trx-id setting trick is way better - it may look weird, but it's "loud and clear", there will be no problem figuring out what (and why) you wrote it there.
Following up on what I would consider a reasonable solution
Modify the TransactionAPIExecutor constructor such that it contains a new field, transaction_id, which can be None or str (basically an Optional[str]). In case it is a str, the constructor should no longer send a request to /_api/transaction/begin, but set the _id property directly and check the status() of the transaction in order to validate it really exists.
The same parameter should be added to the TransactionDatabase, which would forward it to the executor. This is straight forward.
The StandardDatabase should get a fetch_transaction method, which takes the transaction ID and returns a TransactionDatabase. I'm suggesting fetch_transaction because it implies that a transaction may (or may not) be there, rather than continuing one (which is not necessarily "paused").
Testing
Introduce a test case test_transaction.py, something simple, just to check that we're able to use both the initial transaction and the "continued" object.
def test_transaction_fetch(db, col, docs):
txn_db = db.begin_transaction(write=col.name)
txn_col = txn_db.collection(col.name)
txn_db2 = db.fetch_transaction(txn_db.transaction_id)
# insert some documents using both txn's
# ...
Docs
A small edit in _transaction.rst` would be great to showcase how fetch_transaction is supposed to be used.
I'm ready to implement the above. Or, if you want to give it a go, I'm perfectly fine with that, but don't feel pressured, I'm just mentioning since you offered.
Let me know what you think.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

I've opened a PR @apetenchea.
I did consider something like this as well:
request = Request(
    method="get",
    endpoint=f"/_api/transaction/{transaction_id}",
)
resp = self._conn.send_request(request)

if not resp.is_success:
    raise TransactionInitError(resp, request)

result = resp.body["result"]

if result["status"] != "running":
    raise TransactionInitError(resp, request)

self._id = transaction_id

My intention was to prevent a user from 'continuing' a transaction that is already committed or aborted, which would be mostly pointless. However, since the response from the API when fetching the status is 200 OK, raising a TransactionInitError and feeding it the response produced results that would be confusing to the end user. I also realized that perhaps checking the status of a transaction in an external system (that may be committed already) could be useful in some niche cases.
As a sidenote, I noticed that the docs on contributing that are present in the sphinx documentation appears to be outdated. I had to follow the contribution guidelines directly in the repository to get anywhere.


State doesn't keep "color.use" on refresh

Steps to reproduce
Open bubble chart
Open state panel, note that marker.color.use is “property”
Click color, then select “constant”
Note that marker.color.use is now “constant”
Refresh the page
Note that marker.color.use is now again “property”
use = “constant” didn’t save-restore in state successfully
Expected result:
marker.color.use be “constant” after refresh
might be a state persistency problem or validation problem
if it's validation i can fix it, if it's state persistency please help
upd: i checked validation, and it looks like use = “property” before any validation starts

#1180 might be a root cause

Changing colour still breaks things after fixing #1180.

this issue is not related to #1180

https://github.com/Gapminder/vizabi/pull/1197


add global CampaignRegistry

PR sent for .luacheckrc_header, as recommended

Is there documentation on CampaignRegistry?
It's not listed: https://fantasygroundsunity.atlassian.net/wiki/spaces/FGCP/pages/996644556/API+-+Packages

This is the only place I know of that it is mentioned.  I found it from the ruleset source code.  It is associated with the database file in the campaign root folder named CampaignRegistry.lua
https://fantasygroundsunity.atlassian.net/wiki/spaces/FGCP/pages/996644496/Ruleset+-+Scripting#Registries

How interesting!
It looks like there's also a global registry?
I didn't realize that all of that was being stored in Lua tables...

Oh, I didn't know about the GlobalRegistry.lua, lol.  Looks like that file is in the root data folder for fantasy grounds.  Mine just has sound, dice, and death splat settings.  I think that normally these files are accessed using prebuilt functions, but I had a specific purpose where the function wasn't working and had to workaround by accessing the data directly.


Various bugfixes

Fixes broken link in content
Makes section handling more resilient
Increases timeout for database creation

Why did we need to update the version? As we have only released 1.0.0 yet, I would have thought that we would stick with 1.1.0 until released.

Let's do that. I will revert the version change.


Feature Request: Android battery drain measurement

Android provides an interface to access battery counters to measure battery drain
https://source.android.com/devices/tech/power/device
https://perfetto.dev/docs/data-sources/battery-counters
Is it possible for devlib to support this API to help with measuring power consumption of an Android device?
Thanks.

This looks like it could be interesting to investigate.
Would the aim here to be to expose a similar set of properties as the "battery fuel gauge"?

This looks like it could be interesting to investigate.
Would the aim here to be to expose a similar set of properties as the "battery fuel gauge"?

I believe that, on some devices at least, battery status may already be exposed via sysfs and can be collected via existing instrumentation. However, it may be worth adding a probe that uses the standard Android API as a more portable solution...

I believe that, on some devices at least, battery status may already be exposed via sysfs and can be collected via existing instrumentation. However, it may be worth adding a probe that uses the standard Android API as a more portable solution...

Sorry I should have mentioned the goal of why I'm requesting this too!
The idea is that being able to use WorkloadAutomation to run performance test on Android devices and use this interface to collect power measurements to see the impact. Ultimately helping us do A/B comparison after a code change. Since instrumenting the device for power measurement is not easy, using this API hopefully can give us something generic to work with over a bigger range of devices/platforms.

Sorry I should have mentioned the goal of why I'm requesting this too!
The idea is that being able to use WorkloadAutomation to run performance test on Android devices and use this interface to collect power measurements to see the impact. Ultimately helping us do A/B comparison after a code change. Since instrumenting the device for power measurement is not easy, using this API hopefully can give us something generic to work with over a bigger range of devices/platforms.

Looking into this a bit more it seems the simplest way to get basic battery stats from the system would be via dumpsys battery which gives an output similar to following which we could expose:
Current Battery Service state:
  AC powered: false
  USB powered: true
  Wireless powered: false
  Max charging current: 500000
  Max charging voltage: 5000000
  Charge counter: 2383748
  status: 3
  health: 2
  present: true
  level: 93
  scale: 100
  voltage: 4089
  batteryPct=(level*100)/scale

Would these, or a subset of, metrics be sufficient for your usecase?
I'm also wondering if this is to be used for A/B comparison, would you expect to ensure the device had the same starting state and then to take a reading at the start of a job and again at the end of the run comparing the decrease in battery power?

Looking into this a bit more it seems the simplest way to get basic battery stats from the system would be via dumpsys battery which gives an output similar to following which we could expose:
Current Battery Service state:
  AC powered: false
  USB powered: true
  Wireless powered: false
  Max charging current: 500000
  Max charging voltage: 5000000
  Charge counter: 2383748
  status: 3
  health: 2
  present: true
  level: 93
  scale: 100
  voltage: 4089
  batteryPct=(level*100)/scale

Would these, or a subset of, metrics be sufficient for your usecase?
I'm also wondering if this is to be used for A/B comparison, would you expect to ensure the device had the same starting state and then to take a reading at the start of a job and again at the end of the run comparing the decrease in battery power?

I need to revisit the dumpsys battery. But it's not what I'm after here.
If you try an app like Battery Charging Monitor: https://play.google.com/store/apps/details?id=com.tofabd.batteryanalyzer&hl=en_GB&gl=US
You'll see that it gives real time reading of the current drawn from the battery. So my idea was to reset the counter, start an experiment and dump the collected samples of current in experiment A. Then modify the code and re-run experiment B. If my modification in experiment B saves power, the Sum of current samples in B should be smaller than A.
One issue is when the USB cable is connected, it has to be set not to charge. Though it's not a requirement. I ran an experiment of running Geekbench when it's charging and I think it should be usable. Ie: when USB is not charing I see the current in the negative ~ -500uA. When it is charging it is at ~ +1250uA then drops to ~ +700uA when Geekbench is running. So we can observe the difference.
In my example, if the device is charging then the Sum of Exp. B should be higher to indicate power save (it has charged the battery more than Exp. A).
If we can disable charging when running a test that'd be better but I need to look up how feasible is this.
Hopefully this makes sense?

I need to revisit the dumpsys battery. But it's not what I'm after here.
If you try an app like Battery Charging Monitor: https://play.google.com/store/apps/details?id=com.tofabd.batteryanalyzer&hl=en_GB&gl=US
You'll see that it gives real time reading of the current drawn from the battery. So my idea was to reset the counter, start an experiment and dump the collected samples of current in experiment A. Then modify the code and re-run experiment B. If my modification in experiment B saves power, the Sum of current samples in B should be smaller than A.
One issue is when the USB cable is connected, it has to be set not to charge. Though it's not a requirement. I ran an experiment of running Geekbench when it's charging and I think it should be usable. Ie: when USB is not charing I see the current in the negative ~ -500uA. When it is charging it is at ~ +1250uA then drops to ~ +700uA when Geekbench is running. So we can observe the difference.
In my example, if the device is charging then the Sum of Exp. B should be higher to indicate power save (it has charged the battery more than Exp. A).
If we can disable charging when running a test that'd be better but I need to look up how feasible is this.
Hopefully this makes sense?

One issue is when the USB cable is connected, it has to be set not to charge. Though it's not a requirement.

I think that this will be a requirement to be able to get comparable readings as the level of the battery can greatly affect the amount of current being drawn, if the battery is nearly empty you are likely to obtain much larger values that if running with the battery nearly full and this could greatly skew the results. I would interested to see if you repeated your test at differing charge levels whether this would be shown via the application?
On this topic I have found there are some applications that attempt to control the charging behavior however it looks like this is not standardised and they attempt to do this by maintaining a list of common nodes exposed via sysfs. E.g. from: https://github.com/sriharshaarangi/BatteryChargeLimit/blob/ff7360c6048475f1843628c6d47a79313e87f6af/app/src/main/res/raw/control_files.json
/sys/class/hw_power/charger/charge_data/enable_charger
/sys/class/power_supply/ac/charging_enabled
/sys/class/power_supply/battery/batt_slate_mode
/sys/class/power_supply/battery/battery_charging_enabled
...

If you try an app like Battery Charging Monitor ...

I see, I think I understand your aim better now thanks. I had a look at similar application "Ampere" to see how they were managing to read the current value and again it looks like they are attempting to file and read the value as exposed via sysfs.
(https://forum.xda-developers.com/t/app-4-0-3-ampere-current-meter.3040329/#post-59086006)
The list of currently scanned file interfaces:
    /sys/class/power_supply/battery/batt_current
    /sys/class/power_supply/battery/batt_current_adc
    /sys/class/power_supply/battery/batt_current_now
    /sys/class/power_supply/battery/BatteryAverageCurrent
    ...

Android provides an interface to access battery counters to measure battery drain

I believe that, on some devices at least, battery status may already be exposed via sysfs and can be collected via existing instrumentation. However, it may be worth adding a probe that uses the standard Android API as a more portable solution...

I tried printing these properties out from the API on my nexus 5 and received the following values:
Remaining energy = -9223372036854775808
Average energy = -9223372036854775808
Current energy = -771174

The "current energy" value looked it matches what was reported by the sysfs node but the others look like garbage values so I'm not sure how reliable these would be on other devices and the documentation only seems to talk about nexus devices. Although as the applications I found are still relying on sysfs rather than using the android API I wonder if that could be an indication...

So overall it seems to me that the aim here to try and minimise additional influences would be to ensure that battery charging was disabled, and then to sample the reported voltage and current values (via some mechanism) to estimate the amount of energy consumed over the an amount of time. I suspect these reading may still be rather inaccurate compared to hardware instrumentation however it might be enough to provide A/B comparisons for multiple runs on the same device.
Does that seem reasonable?

One issue is when the USB cable is connected, it has to be set not to charge. Though it's not a requirement.

I think that this will be a requirement to be able to get comparable readings as the level of the battery can greatly affect the amount of current being drawn, if the battery is nearly empty you are likely to obtain much larger values that if running with the battery nearly full and this could greatly skew the results. I would interested to see if you repeated your test at differing charge levels whether this would be shown via the application?
On this topic I have found there are some applications that attempt to control the charging behavior however it looks like this is not standardised and they attempt to do this by maintaining a list of common nodes exposed via sysfs. E.g. from: https://github.com/sriharshaarangi/BatteryChargeLimit/blob/ff7360c6048475f1843628c6d47a79313e87f6af/app/src/main/res/raw/control_files.json
/sys/class/hw_power/charger/charge_data/enable_charger
/sys/class/power_supply/ac/charging_enabled
/sys/class/power_supply/battery/batt_slate_mode
/sys/class/power_supply/battery/battery_charging_enabled
...

If you try an app like Battery Charging Monitor ...

I see, I think I understand your aim better now thanks. I had a look at similar application "Ampere" to see how they were managing to read the current value and again it looks like they are attempting to file and read the value as exposed via sysfs.
(https://forum.xda-developers.com/t/app-4-0-3-ampere-current-meter.3040329/#post-59086006)
The list of currently scanned file interfaces:
    /sys/class/power_supply/battery/batt_current
    /sys/class/power_supply/battery/batt_current_adc
    /sys/class/power_supply/battery/batt_current_now
    /sys/class/power_supply/battery/BatteryAverageCurrent
    ...

Android provides an interface to access battery counters to measure battery drain

I believe that, on some devices at least, battery status may already be exposed via sysfs and can be collected via existing instrumentation. However, it may be worth adding a probe that uses the standard Android API as a more portable solution...

I tried printing these properties out from the API on my nexus 5 and received the following values:
Remaining energy = -9223372036854775808
Average energy = -9223372036854775808
Current energy = -771174

The "current energy" value looked it matches what was reported by the sysfs node but the others look like garbage values so I'm not sure how reliable these would be on other devices and the documentation only seems to talk about nexus devices. Although as the applications I found are still relying on sysfs rather than using the android API I wonder if that could be an indication...

So overall it seems to me that the aim here to try and minimise additional influences would be to ensure that battery charging was disabled, and then to sample the reported voltage and current values (via some mechanism) to estimate the amount of energy consumed over the an amount of time. I suspect these reading may still be rather inaccurate compared to hardware instrumentation however it might be enough to provide A/B comparisons for multiple runs on the same device.
Does that seem reasonable?

Please don't gate this on the ability to stop USB charging. There are many ways of avoiding them, one of them is just with adb over wifi. Let's limit the scope of this to just battery draining. The documentation can include a warning about the battery not being charged while using the instrument.

I think that this will be a requirement to be able to get comparable readings as the level of the battery can greatly affect the amount of current being drawn, if the battery is nearly empty you are likely to obtain much larger values that if running with the battery nearly full and this could greatly skew the results. I would interested to see if you repeated your test at differing charge levels whether this would be shown via the application?

As Javi already pointed out, the USB issue isn't a show stopper.
I will try to schedule an experiment next week to see how current reading would differ depending on battery charge. Accuracy can be an issue but I think the limitation it will impose is on the length of the experiment one can conduct only. If each experiments take few mins I'd expected A/B comparison to be fairly representative. But if each one takes 2 hours, then a total run time of 4 hours could be suffering from the accuracy problems, yes. I will try to collect some data.

The "current energy" value looked it matches what was reported by the sysfs node but the others look like garbage values so I'm not sure how reliable these would be on other devices and the documentation only seems to talk about nexus devices. Although as the applications I found are still relying on sysfs rather than using the android API I wonder if that could be an indication...

If the API is not reliable that would be a bummer actually. I thought these apps are using it :-/

Does that seem reasonable?

Yes. In a lot of cases it'll help doing more tests and measurements on desk without special instrumentation; especially in the WFH environment we're in now. IMO it should be good enough to run 10 mins test for experiment A, then flash new kernel and run experiment B for another 10 mins and see if things are any better.

I will try to schedule an experiment next week to see how current reading would differ depending on battery charge. Accuracy can be an issue but I think the limitation it will impose is on the length of the experiment one can conduct only. If each experiments take few mins I'd expected A/B comparison to be fairly representative. But if each one takes 2 hours, then a total run time of 4 hours could be suffering from the accuracy problems, yes. I will try to collect some data.

I've put together a WIP instrument to try and help with this investigation [1] [2]. It aims to search for a valid sysfs node for current readings and polls this along with dumpsys to obtain voltage readings. With this it is usually just about able to distinguish between an idle workload and running dhrystone but the readings are not at all consistent between multiple runs. Unfortunately I currently only have a nexus 5 to test this on so it would be interesting to see if this behavior also applies to other devices.
The instrument will save it's raw readings in the workload directory and adds a summary with the derived energy metrics to estimated the total energy and average battery power.
At it's very best the outputs looks something like the following. The jobs are 30 seconds each, have battery charging manually disabled and uses a fixed delay of 30 seconds between each job to try and allow time for the readings to update between each job.
wk1, Dhrystone - 1 Thread,1,battery_total_energy,22.043696616739986,joules
wk1, Dhrystone - 1 Thread,1,battery_average_power,0.7399434510000003,watts
..
wk2,Dhrystone  - 4 Thread,1,battery_total_energy,32.05555683436006,joules
wk2,Dhrystone  - 4 Thread,1,battery_average_power,1.0146634870384617,watts
...
wk3,idle,1,battery_total_energy,0.8461401516782534,joules
wk3,idle,1,battery_average_power,0.045117365812500004,watts

wk1,Dhrystone - 1 Thread,2,battery_total_energy,5.573439491921011,joules
wk1,Dhrystone - 1 Thread,2,battery_average_power,0.18477347599999996,watts
...
wk2,Dhrystone - 4 Thread,2,battery_total_energy,20.866000051942436,joules
wk2,Dhrystone - 4 Thread,2,battery_average_power,0.6890252682000001,watts
...
wk3,idle,2,battery_total_energy,0.8297014357797509,joules
wk3,idle,2,battery_average_power,0.044204215,watts

As can be seen, the results for the same workload appear to vary hugely so it looks like this still requires further digging to see if these readings would be able to produce any useful results.
Please let me know what behavior you observe in your experiments.
[1] https://github.com/ARM-software/workload-automation/pull/1159
[2] https://github.com/ARM-software/devlib/pull/524

Thanks a lot Marc. I was expecting to get current readings. Next week I will dedicate more time to run experiments on Pixel 4 and report back.

Ok great thanks.
Just FYI the raw readings are saved in the job directory to inspect directly, an example is show below with the output normalized using the provided scaling factors.
battery_current,	battery_percent,battery_power,	battery_voltage,	timestamp_time
0.011596,			21.0,		0.043624152,		3.762,<PHONE_NUMBER>.199288
0.011596,			21.0,		0.043624152,		3.762,<PHONE_NUMBER>.415279
0.964349,			21.0,		3.627880938,		3.762,<PHONE_NUMBER>.6282794
0.964349,			21.0,		3.627880938,		3.762,<PHONE_NUMBER>.8619392
0.964349,			21.0,		3.627880938,		3.762,<PHONE_NUMBER>.0887477
0.964349,			21.0,		3.627880938,		3.762,<PHONE_NUMBER>.3295774
...

Sorry for the delayed response. I grabbed your PRs and managed to have a run on Pixel 4 using hackbench. I did 10 runs without a delay between the runs.

       battery_current  battery_percent  battery_power  battery_voltage
count        95.000000        95.000000      95.000000        95.000000
mean          0.802582         9.368421      -2.781394         3.469716
std           0.098025         0.799636       0.315172         0.072881
min           0.377187         8.000000      -4.192935         3.336000
25%           0.769843         9.000000      -2.817744         3.426000
50%           0.799375         9.000000      -2.754444         3.438000
75%           0.815937        10.000000      -2.690218         3.481000
max           1.256875        11.000000      -1.364285         3.617000

I think the numbers look sane. I just found 1 second minimum sampling time is too coarse, no?
I did run some experiments with hackbench and reading current_avg from sysfs and the numbers looked sane to me. I used both 10ms and 100ms in that case as I was running a script on the target. Not sure in your case you issue a adb command every 1second or you run something on the target. The latter might be more reliable.
I was talking to the device via TCP by the way.

I just found 1 second minimum sampling time is too coarse, no?
On my device I found that the raw values only updated every 3 or 4 seconds, were you able to obtain more precise data from you device?

I did run some experiments with hackbench and reading current_avg from sysfs and the numbers looked sane to me.
Interesting, do you have a comparisons set of runs using an idle workload to compare the difference?

Not sure in your case you issue a adb command every 1second or you run something on the target. The latter might be more reliable.
Yes the current implementation polls with an adb command which I agree is not ideal, for the proper implementation it would definitely be better to run purely on the device similar to how the file poller operates.

Yes, using my own script I collected data using 10ms and 100ms sampling rate. The 10ms caused the baseline power to be higher since more work is done in the background. This picture is from 5 runs of hackbench with 15 seconds between each run. Note that each run is collected independently and aggregated at the end, hence the slope between each run

The below is a capture when the device is idle. Note in both tests I hold /sys/power/wake_lock otherwise deep sleep causes issues. I certainly had less variations in the past compared to this run I just captured

My script is very simple
#!/bin/sh
set -eu

OUTPUT=$1

echo "timestamp,current" > $OUTPUT
while true
do
	echo "$(date +%s),$(cat /sys/class/power_supply/battery/current_avg)" >> $OUTPUT
	usleep 100000
done

I did these 3 following 1hour runs using 100, 50 and 10 ms sampling rates respectively. Again I hold the wake_lock during these runs.

Thanks for doing those run, they look a lot more promising than what I had on my device, although I guess that could also be down to the age difference in devices.
Did you have a chance to test out the instrument in the PR's to see if the methodology there is valid and whether that something that would be of use? Or are you looking for something more low level for your own processing?
I'm thinking for either approach we could export an attribute of the device to try and auto discover the path of the current node which would allow the use with custom scripts or via the poller instrument for direct access to the counters, and then we'll need a similar attribute for voltage etc.

Did you have a chance to test out the instrument in the PR's to see if the methodology there is valid and whether that something that would be of use? Or are you looking for something more low level for your own processing?

Yes I did. The result in https://github.com/ARM-software/devlib/issues/522#issuecomment-800699045 are from your PRs. I used my own script to see how usable it is to do ms granularity sampling only. If I could have done that with your PR then sorry I missed it.
The goal is really to be able to use that in absence of proper hardware instrumentation. I admit my experience of setting up wa with hardware instrumentation is close to none. But assuming there's a standard output from using acme cape or monsoon for instance, it'd be good if battery_mon produces similar output so that other tools built on top can transparently use it. I'm thinking mainly wltests in LISA.
https://github.com/ARM-software/lisa/tree/master/tools/wltests
Generally with this approach I can think of 2 potential limitations:

Sensitivity. We might not be able to detect a small difference in energy consumption between A/B. By how much more experiments is required.
I think workloads that are relatively busy will be more reliable than the ones that have significant amount of time in idle.

For 2, imagine I have improved the code such that it will allow the device to spend more time in deeper idle state, then the measurement noise could interfere with the ability to detect this.
We might be able to mitigate against that if we ensure the poller script/instrument is attached to "Background" cgroup. This might reduce its impact since they are limited with the amount of resources they can consume.
Thinking out loudly mostly :-)

Yes I did. The result in #522 (comment) are from your PRs. I used my own script to see how usable it is to do ms granularity sampling only. If I could have done that with your PR then sorry I missed it.

Ah right yes that makes sense, no I hadn't expected that granularity to work, so that is something I can modify to allow.

But assuming there's a standard output from using acme cape or monsoon for instance, it'd be good if battery_mon produces similar output so that other tools built on top can transparently use it. I'm thinking mainly wltests in LISA.

We have a standard instrument output format in the form of a MeasurementsCSV which the PR should adhere to and is what allows the DerivedEnergy instrument to operate on the collected output. Although thanks for mentioning wltests, I'm not familiar with what they're expecting so will take a look.

Sensitivity. We might not be able to detect a small difference in energy consumption between A/B. By how much more experiments is required.

Agreed, and by the looks of it, it seems that it would vary largely depending on the particular device in question.

For 2, imagine I have improved the code such that it will allow the device to spend more time in deeper idle state, then the measurement noise could interfere with the ability to detect this.

Yes that is a good point, I think the overhead is going to be the other limitation of this approach, it certainly requires more thought but I guess the first step would be optimize this approach to minimize it's footprint.

I got a pixel 5 setup now too. I had a hackbench run with your PR but the numbers look wrong to me :-/

I have tried with my script wtih 100ms sampling rate, it looks okay(ish) but the implementation of /sys/class/power_supply/battery/current_avg is not as good as in pixel 4. The value is only updated at fixed intervals

Note that on pixel 5 the sysfs returns negative value compared to a positive value on p4. I did all my runs without usb cable plugged, so definitely weren't charging.
On pixel 5 /sys/class/power_supply/bms/current_avg returns an identical value to the one read from 'battery' folder but positive. I hope this makes sense.

In your PR, do you try to use the BatteryService/BatteryManager Java API too or just try to find the best sysfs node to read the values from? Not sure if they're any good by the way, just wondering

In your PR, do you try to use the BatteryService/BatteryManager Java API too or just try to find the best sysfs node to read the values from?

I'm currently just attempting to find the best sysfs node, from my (albeit limited) testing the number reported from both were exactly the same so I thought reading the sysfs node would prevent the requirement for running a separate java application just to call the API. Have you observed any different behavior?

Note that on pixel 5 the sysfs returns negative value compared to a positive value on p4. I did all my runs without usb cable plugged, so definitely weren't charging.
On pixel 5 /sys/class/power_supply/bms/current_avg returns an identical value to the one read from 'battery' folder but positive.

That is interesting how is is reported differently on the same device, however to try and combat this as part of the PR when detecting the node it will invert the value as required so that the value is reported (this currently assumes that the device is discharging when initialization takes place however this could be post processed instead if it is more reliable.)

had a hackbench run with your PR but the numbers look wrong to me
I have tried with my script wtih 100ms sampling rate, it looks okay(ish) but the implementation of /sys/class/power_supply/battery/current_avg is not as good as in pixel 4. The value is only updated at fixed intervals

Thanks for trying these out, and it is a shame to see that the later pixel phone does not appear to provide as good. Also I might be missing the obvious but which numbers are looking out of place to you for the PR chart vs from your script? The voltage seems about right at 4.xV and the current looks to be similar to your script's output (both unfortunately varying quite a bit more compared to your pixel 4 runs) after it has been inverted to provide a positive value, although it does seem to have a higher average value...

Have you observed any different behavior?

Not not really. And I don't know if it's worth the effort or not to be honest. That API is supposed to go via the HAL, so its implementation will be platform specific. Maybe they just hook it up to the sysfs node.. Who knows. Beside on AOSP builds I don't know if we get proper HAL implementation there either. I doubt it, so it could be a dead end.

Thanks for trying these out, and it is a shame to see that the later pixel phone does not appear to provide as good.

Indeed. Pixel 4 reading the current_avg from sysfs seemed to be decent. But on P5 it's not as good. Potentially I might be able to tinker with the source code to make it behave better..

Also I might be missing the obvious but which numbers are looking out of place to you for the PR chart vs from your script?

Or it could be me who missed something obvious too :-) I got confused because the current values are hovering around 1. But I looked back at the pixel 4 plot in https://github.com/ARM-software/devlib/issues/522#issuecomment-800699045 and it seems you normalize the values and I got caught out with this. So it's me who missed something here hehe :)
Thanks!

Hi. Just checking if there's any plan to handle this request?

Hi @qais-yousef, Apologies I have not had time to make much progress on this since the last update, however yes this is still on my todo list to complete the investigation.

No worries and thanks for the update


VIDEOJS: WARN: Unexpected adsready event

Hello,
I am trying to request Ad at the end of the content:
this.videojs.on('ended', function() {
    this.ima.initializeAdDisplayContainer()
    this.ima.setContentWithAdTag(null, this.adTagUrl, true)
    this.ima.requestAds()
});

but I am getting this: VIDEOJS: WARN: Unexpected adsready event and nothing to play.
any advice? thanks

Hi,
The plugin doesn't support requesting ads at any point in content, it is tied to the life-cycle of content playback. If you want to play a post-roll at the end of content, you can provide a VMAP tag to the plugin with a post roll in it, such as this sample ad, and it will automatically play at the end of content.

Hi,
may be you should listen contentended instead of ended
this.videojs.on('contentended', function() {
    this.ima.initializeAdDisplayContainer()
    this.ima.setContentWithAdTag(null, this.adTagUrl, true)
    this.ima.requestAds()
});

@ypavlotsky thanks for the information, currently we are trying to prevent the ending of the content so we can make ad-requests whenever we want, by removeEventListener of related events.
@Pong420 thank you for the help, I have tried this before, the same behavior.

Hi @jehadnasser ,
Like @ypavlotsky was saying, the plugin does not support requesting ads during content playback.
What is it you are trying to achieve from requesting ads during content playback?

Hi @Kiro705
I am no longer in the Ad business, but as I remember I was trying to request a new ad once the content ends but ending the content means no possibility to make an ad request.


Primary Contact can be the same on two Household Accounts. Bad.

Repro steps:

go to Account Tab
select NEW -> create a household account
Name the account
Select a Primary Contact (you are forced to use the lookup field, to choose a pre-existing contact, see screenshot)

Press Save

Expected Behavior: We expect that someone who is a primary contact on another account will NOT be able to be selected for this new account.
Actual Behavior: When we select a contact who is primary on another account, they are also saved as primary on this account, even though they do not belong to this household.

**lurch: add

Tracking W-019580

also see #2631

Salesforce has determined that this issue will not be explored further.


Moving the fh-deploy queue to package.json

It would be great if the "queue" array that's required by the fh-deploy config file could be in version control. That queue only needs to change when new files are added or removed, and it's a huge pain to have to rebuild it when you're cloning a project.
It doesn't make sense to have 2 different files, one for credentials that will be gitignored and one for a queue that will be tracked, but maybe it makes sense for fh-deploy to look at the "files" property in the package.json file rather than its own config file?
That property is primarily used to tell npm how to publish a package, but I think it generally makes sense for our use case since our version of "publishing" is just pushing to a Wordpress server.
This obviously would be mostly a change to the fh-deploy package, but we would have to make changes in this repo as well.
@SaFrMo thoughts?

Yep, I like that a lot! This way, the .deployrc file will just be login credentials, which makes perfect sense. I'll make the changes to fh-deploy first, then update here. Great note!

Fixed in f006a8d!
No examples are included, duplicated, renamed, etc. any more because they're not needed. The first time you run npm run deploy, you'll be prompted for some information that automatically creates a config file for you, which is .gitignored and uses the files in the package.json files property by default. This should remove the need for any manual setup of login credentials.
Marking closed - let me know if there are any questions!


undefined symbol: _PyThreadState_UncheckedGet

Tried to install borgbackup on python 3.5.0 (for testing using the minimum 3.5 version):
==> wheezy64: Traceback (most recent call last):
==> wheezy64:   File "setup.py", line 80, in <module>
==> wheezy64:     import Cython.Compiler.Main as cython_compiler
==> wheezy64:   File "/home/vagrant/.pyenv/versions/3.5.0/envs/borg-env/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 28, in <module>
==> wheezy64:     from .Scanning import PyrexScanner, FileSourceDescriptor
==> wheezy64: ImportError: /home/vagrant/.pyenv/versions/3.5.0/envs/borg-env/lib/python3.5/site-packages/Cython/Compiler/Scanning.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet
==> wheezy64:
==> wheezy64: During handling of the above exception, another exception occurred:
==> wheezy64:
==> wheezy64: Traceback (most recent call last):
==> wheezy64:   File "setup.py", line 126, in <module>
==> wheezy64:     raise ImportError('The GIT version of Borg needs Cython. Install Cython or use a released version.')
==> wheezy64: ImportError: The GIT version of Borg needs Cython. Install Cython or use a released version.
==> wheezy64: Obtaining file:///vagrant/borg/borg
==> wheezy64:     Complete output from command python setup.py egg_info:
==> wheezy64:     Traceback (most recent call last):
==> wheezy64:       File "/vagrant/borg/borg/setup.py", line 80, in <module>
==> wheezy64:         import Cython.Compiler.Main as cython_compiler
==> wheezy64:       File "/home/vagrant/.pyenv/versions/3.5.0/envs/borg-env/lib/python3.5/site-packages/Cython/Compiler/Main.py", line 28, in <module>
==> wheezy64:         from .Scanning import PyrexScanner, FileSourceDescriptor
==> wheezy64:     ImportError: /home/vagrant/.pyenv/versions/3.5.0/envs/borg-env/lib/python3.5/site-packages/Cython/Compiler/Scanning.cpython-35m-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet
==> wheezy64:
==> wheezy64:     During handling of the above exception, another exception occurred:
==> wheezy64:
==> wheezy64:     Traceback (most recent call last):
==> wheezy64:       File "<string>", line 20, in <module>
==> wheezy64:       File "/vagrant/borg/borg/setup.py", line 126, in <module>
==> wheezy64:         raise ImportError('The GIT version of Borg needs Cython. Install Cython or use a released version.')
==> wheezy64:     ImportError: The GIT version of Borg needs Cython. Install Cython or use a released version.

In above case, the only pyenv-installed python is 3.5.0.
I've seen a similar error before in borg (using pyenv to get 3.4.0, 3.5.0, 3.6.0), see there:
https://github.com/borgbackup/borg/issues/3066

duplicate of #1880


Do I need logging into shimo in order to get access to the table data

您的问题
I can browser the data directly in browser using the shimo link. So I wonder if it is necessary to login in the code.

But you need to parse the browser data and you can use cookie to login on browser side

I find that shimo frontend code does not use the api to load the data. So it is undocumented and may change without notice.

It is not difficult to summarize the undocumented api using chrome dev-tools. Many popular projects in web-crawler domain do things like this. But I think such kind of method is not the best option. By the way, shimo official teams seem not to open the application of their SDK token. This is the same like lingxi, which you have to contact their staff personally to apply the token. This kind of difficulty set the obstacle for developers to contribute to project like this.

Yes, actually in wuhan2020, shimo is our sponsor from the first day, it's not a totally open platform.


Fix for mixed setTimeout/clearTimeout methods

Noticed that I was unable to use clearTimeout but inly when running through devtool, it looks like we need to shim both setTimeout and clearTimeout otherwise we get the setTimeout method from node and the clearTimeout method from the browser.
Maybe there is a better place to setup protection for clearTimeout/clearInterval? but this seems to work for me.
Thats not fun 👎

Thanks! :smile:


Makefile improvements

Extracted from @stevekuznetsov's rebase PR plus some extensions:

require ko and kind
separate image building into its own target
better error logic

/lgtm
/approve

/retest


⚠ Bofrost.at has degraded performance

In 524d9a8, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 994 ms

Resolved: Bofrost.at performance has improved in 9b5e922.


Enable client scripts.

This pull request enables cascade client (CPP) script in the following style:
# This is a script for C/C++ cascade client
# 1 - List members
list_members
# 2 - Create object pool
create_object_pool /some/object/pool VCSS 0
# 3 - put into object pool
op_put /some/object/pool/key value

This approach is more efficient than bash scripting because the client does not need to connect/disconnect for each command.

I've smoke-tested it for regression test.


Refactor to invoke and call methods instead of invoke / invokeUni

See #273

[x] Uni groups
[ ] Multi groups
[ ] Embrace new methods in tests

@gavinking you can have a look.

Yep I will look at it today.

@jponge LGTM!

Thanks @gavinking!

@cescoffier I propose not to make too much effort on pleasing codecov right now, we know we need to refactor and cleanup the test suite.


Fix #277, avoid unexpected ScalaDocChecker warning for implicit def

Fix #277

#277 seems to be scalariform's bug.
https://github.com/scalastyle/scalastyle/blob/47bae5da4e02fdc4e2efe437ab493617c8ddb68c/src/main/scala/org/scalastyle/scalariform/ScalaDocChecker.scala#L106-L114
token.associatedWhitespaceAndComments in findScalaDoc normally return HiddenToken contains Whitespace (before ScalaDoc comment) token and MULTILINE_COMMENT token, and we can acquire commentOffset checking the Whitespace length .
But when checking scaladoc for implicit function definition, token.associatedWhitesaceAndComments return HIddenToken contains only MULTILINE_COMMENT token, not contains Whitespace token.
So, when checking scaladoc for implicit function definition HiddenTokens(ht.tokens.takeWhile(_.token != commentToken)).rawText returns empty string and commentOffset must be 0, and consequently ScalaDoc.apply(...) have wrong indentStyle and cause unexpected warnings like #277.
I don't understand why scalariform parser return such result, and fix the issue by stop depending on the non-stable scalariform parser's parse result.

Codecov Report

Merging #285 into master will not change coverage.
The diff coverage is 0%.

@@          Coverage Diff          @@
##           master   #285   +/-   ##
=====================================
  Coverage       0%     0%
=====================================
  Files          59     59
  Lines        1451   1451
  Branches      139    144    +5
=====================================
  Misses       1451   1451

Impacted Files
Coverage Δ

...a/org/scalastyle/scalariform/ScalaDocChecker.scala
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93acfba...8528e36. Read the comment docs.

Great. Thanks!


[battery_plus_macos] Add integration test

Battery Plus macOS is currently missing integration tests.
Objectives

[ ] Add test cases for getBatteryLevel
[ ] Add test cases for onBatteryStateChanged

The e2e tests seem to be located in https://github.com/fluttercommunity/plus_plugins/tree/main/packages/battery_plus/test, which already checks for batterylevel. Have you tried to run that test on MacOS?
(also, the test location is wrong, as I reported in https://github.com/fluttercommunity/plus_plugins/issues/28)


Can't detect gesture when inside stack widget.

When I wrap xgesture widget with stack widget, The xgesture widget can't detect any gesture.

Could you please provide your code snippet or at least an example code that demonstrating the issue?


How/are people using 'guides' content on wellcomelibrary.org?

To inform our work on content development for new researchers, we need some information on how existing content is being used on wellcomelibrary.org.
I'd like to get Analytics data for the past 12 months on usage of the following. This should include:

Pageviews
Users
Time on page
Bounce
Exit

Archive guides

From library website: Anything that falls under this page (including all guides within) https://wellcomelibrary.org/collections/archive-guides/
From archives catalogue: Anything starting with ref AMS/G/
http://archives.wellcomelibrary.org/DServe/dserve.exe?dsqIni=Dserve.ini&dsqApp=Archive&dsqCmd=Overview.tcl&dsqDb=Catalog&dsqSearch=(AltRefNo%3D'AMS%2FG%2F')&dsqNum=50&PF=No&dsqPos=0 (these are pages in the archive catalogue that reference a PDF guide. There are 82 records total. Please ignore any that don't begin with AMS.)

Subject guides
Three listed on this page: https://wellcomelibrary.org/collections/subject-guides/
https://wellcomelibrary.org/collections/subject-guides/introduction-to-popular-science-resources/
https://wellcomelibrary.org/collections/subject-guides/biography-and-family-history/
https://wellcomelibrary.org/collections/subject-guides/introduction-to-mortality-statistics-in-england-and-wales/
Other random guides
https://wellcomelibrary.org/collections/about-the-collections/archives-and-manuscripts/finding-medical-archives-elsewhere/
https://wellcomelibrary.org/collections/about-the-collections/archives-and-manuscripts/finding-recipe-manuscripts-online/

Summary stats including average time on page have been compiled in this Google sheet:
https://drive.google.com/open?id=182Jp6xSBcCwRrXNUfHwAz_o9rc_cGhDSbUjc5ct-dPg

https://wellcomelibrary.org/what-we-do/history-of-wellcome-library/researching-wellcome-work/
https://wellcomelibrary.org/what-we-do/history-of-wellcome-library/wellcome-historical-medical-museum/
Sorry @taceybadgerbrook, I missed these two guides. When you have a moment can you please add them to the sheet? Thank you!

Analysis for all guides now available onhttps://docs.google.com/spreadsheets/d/182Jp6xSBcCwRrXNUfHwAz_o9rc_cGhDSbUjc5ct-dPg/edit?usp=sharing