Parallel execution of one application

I want to run my tests parallel and I have to open three instances of the same application at the same time (but with other configuration of the app, but that's not the point). My code executes three applications (I have executed the WinAppDriver.exe with different ports on the Remote Machine). I use a ThreadLocal WindowsDriver and I want to interact with "removeDriver.get()" with each of the three apps.
All three applications are openend, but when I try to interact with the applications, there are intersections. For example, if I enter text in application 1, 2 and 3, then the text will be entered in application 1 or 2 but not in three. So, it seems that the driver uses the wrong session/application? Maybe I have to switch to the active window of the current used driver? All three applications
lie one above the other.
Here is my code:
protected static ThreadLocal<WindowsDriver> remoteDriver = new ThreadLocal<WindowsDriver>();

    private static Logger log;

    static {
        log = Logger.getLogger(MyClass.class);
    }

    public static void start(String client) {
		try {
            DesiredCapabilities capabilities = new DesiredCapabilities();
            capabilities.setCapability("app", "pathtomyapp" + id);
            capabilities.setCapability("platformName", "Windows");
            capabilities.setCapability("deviceName", "WindowsPC");
            String remoteUrl = "http://" + config.getHost() + ":" + config.getPort() + "/wd/hub";
            log.info("Remote URL: " + remoteUrl);
            WindowsDriver driver = new WindowsDriver<WindowsElement>(new URL(remoteUrl), capabilities);
           	driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
           	remoteDriver.set(driver);
        }catch(Exception e){
        	e.printStackTrace();
        } finally {
        }
    }

    public static void stop() {
    	getDriver().quit();
    }

    public static void enterText(String element, String input) {
    	remoteDriver.get().findElementByAccessibilityId(element).sendKeys(input);
    }

Hi @taddeuz,
While some things can occur in parallel it is not recommended to run multiple tests in this way. The way input (mouse, keyboard) works, it by sending keystrokes as it would from a physical keyboard. So when you sendKeys to an element, what actually happens is that the element is given focus and keys are sent, but if in the middle of sending those keys another element on the system gets focus the rest of the keys will be sent to the element that has focus, just as if someone had take the mouse as you're typing and selected another application. The same is true for mouse interactions, for example when you do a click, the mouse pointer will move to the location of the element and issue a click, just like a user would do.
If you were to run the tests in parallel to three different machines, then that should work, but on the same machine, you will have such conflicts.

So, there is no way to run the tests in parallel on one machine?
With selenium webdriver it is possible to run Tests in parallel with different browsers on one VM. Why its not possible with WAD?
I used winium before implementing WAD. winium has a Session problem with multiple executions, so I switched to WAD - because parallel executions should be possible (according to the wiki and some posts).

So, I have to use an own machine / VM for each application?

@khouzam
So, there is no way to run the tests in parallel on one machine?
With selenium webdriver it is possible to run Tests in parallel with different browsers on one VM. Why its not possible with WAD? Of course, Selenium is an other framework.
I used winium before implementing WAD. winium has a Session problem with multiple executions, so I switched to WAD - because parallel executions should be possible (according to the wiki and some posts).
Is there a way to switch to the application window of used session?

Yes, you can run parallel test on the same machine using winapp driver.
Just make sure you use different user sessions, and different winappdriver ports per session, you can set a user environment variable for the port value.
You get to also have in consideration in your code the session user when closing processes related to the apps you are testing and the winappdriver process.
Regards,
Orlando


Not Working with ERROR in ./node_modules/react-native-svg/lib/module/LocalSvg.js

ERROR in ./node_modules/react-native-svg/lib/module/LocalSvg.js 42:49-107
Module not found: Error: Can't resolve 'react-native/Libraries/Image/resolveAssetSource' in '/home/box/dev/web/node_modules/react-native-svg/lib/module'
Inside package.json
"react": "^17.0.1",
"react-dom": "^17.0.1",
"react-native-svg": "^12.1.0",
"react-native-web": "^0.15.0",
using webppack.config.js
const HtmlWebpackPlugin = require('html-webpack-plugin');
const path = require('path');

module.exports = {
  entry: './src/index.js',
  output: {
    path: path.resolve(__dirname, './release'),
    filename: 'bundle.js',
  },
  plugins: [
    new HtmlWebpackPlugin({
      filename: 'index.html',
      template: './index.html',
    }),
  ],
  module: {
    rules: [
      {
        test: /\.(js|jsx)$/,
        include:[
          path.resolve(__dirname, './src'),
          path.resolve(__dirname, './node_modules/react-native-vector-icons'),
          path.resolve(__dirname, './node_modules/react-native-svg'),
        ],
        use: {
          loader: 'babel-loader',
          options: {
            presets: [
              '@babel/preset-env',
              '@babel/preset-react',
              {
                plugins: [
                  '@babel/plugin-proposal-class-properties'
                ]
              }
            ],
            plugins: ['babel-plugin-react-native-web'],
          },
        },
      },
      {
        test: /\.ttf$/,
        loader: "url-loader", // or directly file-loader
        include: path.resolve(__dirname, "node_modules/react-native-vector-icons"),
      },
      {
        test: /\.png$/i,
        loader: "file-loader", // or directly file-loader
        include: path.resolve(__dirname, "node_modules/react-native-svg"),
      }
    ],
  },
  resolve: {
    alias: {
      'react-native$': 'react-native-web',
    },
  },
};

I have similar issue. after updating to "react-native-svg": "^12.1.0" working properly.

me too using "react-native-svg": "^12.1.0"
can you share webpack config file?

I had the same issue using Storybook, Metro has a config to import files with .android.js and .ios.js extensions and react-native-web adds up with the .web.js extension, which Webpack is not configured to handle. When resolving this file ReactNativeSVG.web.ts, Webpack resolves to the Native version instead of the web version, which is causing the bug.
Webpack has a config resolve.extensions, I was able to solve this by adding .web.js to the top of the list. It should work for you too.

Thanks a lot @renatoalencar for guidance after including .web.js on top of list for webpack.extensions. able to resolve the issue.

Thank you @renatoalencar, that did it!
In the webpack.config.js file, adding the following fixed it for me:
module.exports = ({config}) => {
  // ...
  config.resolve.extensions.unshift('.web.js'); // added this line
  // ...
  return config;
};


unable to import component

when i put the component into ngmodule declarations, i got an error saying 'Unexpected value 'Ng2SimpleAutocomplete' declared by the module 'AppModule'. Please add a @Pipe/@Directive/@Component annotation.'
I'm using the latest version of angular, any idea how this could happen?

I think you might have added Ng2SimpleAutocomplete  at root NgModule that named AppModule.... Sorry I have no idea which problem makes that message.
I just tried this autocomplete module with angular-starter which uses<EMAIL_ADDRESS>and it works well. 'angular starter' uses webpack, so I tested on my own private project which uses angular<EMAIL_ADDRESS><EMAIL_ADDRESS>It works well too.
I think it may be not related to your problem, I figured out this package requires 'tslib' while testing. So I moved it to the dependencies list and published a new version.

+1


Egress Policy v1alpha1 implementation

Describe what you are trying to solve
This proposal summarizes the first alpha version of the Egress Policy feature. Please see https://github.com/vmware-tanzu/antrea/issues/667 for the complete proposal.
In v1alpha1, we require users to manually configure SNAT IPs on the Nodes. In a EgressPolicy, a particular SNAT IP can be specified for the selected Pods, and antrea-controller will publish the EgressPolicy to the Nodes on which the selected Pods run.
There will be some limiations in the first version: encap mode is the only supported traffic mode. Some features and scenarios, e.g. HA, dual-stack and windows are not supported.
Describe how your solution impacts user flows

User configures secondary IPs that can be used as SNAT IPs to Nodes' network interfaces.
User configures EgressPolicy (a CRD API) which selects specific Pods and the IP they should be translated to when accessing external addresses.

Describe the main design/architecture of your solution
API change
An user-facing API will be introduced. The object schema will be like below:
type EgressPolicy struct {
	metav1.TypeMeta `json:",inline"`
	// Standard metadata of the object.
	metav1.ObjectMeta `json:"metadata,omitempty"`

	// Specification of the desired behavior of EgressPolicy.
	Spec EgressPolicySpec `json:"spec"`
}

// EgressPolicySpec defines the desired state for EgressPolicy.
type EgressPolicySpec struct {
	// AppliedTo selects Pods to which the policy will be applied.
	AppliedTo AppliedTo
	// EgressIP specifies the SNAT IP address for the selected Pods.
	EgressIP string
}

// AppliedTo defines the workloads to which a policy is applied.
type AppliedTo struct {
	// Select Pods matched by this selector. If set with NamespaceSelector,
	// Pods are matched from Namespaces matched by the NamespaceSelector;
	// otherwise, Pods are matched from all Namespaces.
	// +optional
	PodSelector *metav1.LabelSelector `json:"podSelector,omitempty"`
	// Select all Pods from Namespaces matched by this selector, as
	// workloads in To/From fields. If set with PodSelector,
	// Pods are matched from Namespaces matched by the NamespaceSelector.
	// +optional
	NamespaceSelector *metav1.LabelSelector `json:"namespaceSelector,omitempty"`
}

Similar to NetworkPolicy, each EgressPolicy needs to be converted by antrea-controller to internal (controlplane) resources to be consumed by agents. This is mainly to avoid redundant Pod watching and group calculation when resolving "AppliedTo".
type EgressPolicy struct {
	metav1.TypeMeta `json:",inline"`
	// Standard metadata of the object.
	metav1.ObjectMeta `json:"metadata,omitempty"`
        // EgressGroup is the name of the EgressGroup to which this policy applies.
	EgressGroup string

	EgressIP string
}

type EgressGroup struct {
	metav1.TypeMeta
	metav1.ObjectMeta
	// GroupMembers is a list of resources selected by this group.
	GroupMembers []GroupMember
}

Control Plane
antrea-controller
antrea-controller watches the policies from Kubernetes API, converts them to the controlplane policies, creates the EgressGroups. Two APIs in the controlplane API group, "EgressPolicy" and "EgressGroup", will provide list, get, and watch interface for agents to consume.
antrea-agent
antrea-agent watches the above controlplane "EgressPolicy" and "EgressGroup" APIs, then:
For each EgressPolicy, it checks whether the EgressIP is configured on the Node it runs on. If yes, it allocates a locally-unique ID (usage mentioned in the "Data plane" section below) for this IP and configures corresponding openflow rules and iptables rules to enforce SNAT for specific traffic. Otherwise it does nothing.
For each Pod in EgressGroup, it checks whether the associated EgressIP is local or not. If local, it configures specific openflow rules to forward the traffic coming from the Pod to the gateway interface with specific mark set. If remote, it configures specific openflow rules to forward the traffic to the tunnel interface with specific tunnel destination set.
Data Plane
(Copied from https://github.com/vmware-tanzu/antrea/issues/667#issuecomment-781818942)
On the Node, antrea-agent will realize the SNATPolicy with OVS flows and iptables rules. If the SNAT IP is not present on the local Node, the packets to be SNAT'd will be tunneled to the SNAT Node using the SNAT IP to be the tunnel destination IP. On the SNAT Node, the tunnel destination IP will be directly used as the SNAT IP.
On the SNAT Node, an iptables rule will be added to perform the SNAT with the specified SNAT IP, but which SNAT IP to use for a given packet is controlled by the OVS flows. The OVS flows will mark a packet that needs to be SNAT'd with a SNAT IP with the corresponding integer ID, and the corresponding iptables SNAT rule matches the packet MARK.
The OVS flow changes include:
table 31
// SNAT flows for Windows
- priority=210 ip,-new+trk,snatCTMARK,from_uplink macRewriteMark,goto:40 (SNAT return traffic)
+ priority=210 ip,-new+trk,snatCTMARK,from_uplink,nw_dst=localSubnet macRewriteMark,goto:40 (SNAT return traffic - remote packets will be handled by L3Fwd flows, so no need to set the macRewrite MAC)

table 70
// Reuse these Windows SNAT flows to skip packets need not SNAT
+priority=200 ip,from_local,nw_dst=localSubnet goto:80
+priority=200 ip,from_local,nw_dst=nodeIP goto:80
+priority=200 ip,from_local,nw_dst=gatewayCTMark goto:80

// Send packets for external network to the SNAT table
+priority=190 ip,from_local goto:71
+priority=190 ip,macRewriteMark mod_dl_dst:gw0_mac,goto:71 (traffic tunneled from remote Nodes)

+table 71 (snatTable. ttlDecTable is moved to table 72)
// Windows flows: load SNAT IP to a register (probably share the endpointIPReg and endpointIPv6XXReg)
priority=200 ip,+new+trk,in_port=local_pods snatRequiredMark(snat_ip),goto:80 (SNAT for local Pods, matching in_ports)
priority=200 ip,+new+trk,tun_dst=snat_ip snatRequiredMark(tun_dst),goto:80 (SNAT for remote Pods, matching tun_dst)
priority=190 ip,+new+trk snatRequiredMark(node_ip),goto:80 (default SNAT IP)

// Linux: mark the packet with an integer ID allocated for each SNAT IP
priority=200 ip,+new+trk,in_port=local_pods mark(snat_id),goto:80 (SNAT for local Pods)
priority=200 ip,+new+trk,tun_dst=snat_ip mark(snat_id),goto:80 (SNAT for remote Pods)

// common: tunnel packets need to SNAT on a remote Node with the SNAT IP to be the outer destination
priority=200 ip,in_port=local_pods mod_dl_src:gw0_mac,mod_dl_dst:vMAC,snat_ip->NXM_NX_TUN_IPV4_DST,goto:72
priority=0 goto_table:80

+table 72 (ttlDecTable)

table 105
// Windows: perform SNAT with the SNAT IP saved in the register
+priority=200 ip,+new+trk,snatRequiredMark ct(commit,table=110,zone=65520,nat(src=snat_ip),snatCTMark)

iptables rules:
iptables -t nat -A POSTROUTING -m mark --mark snat_id -j SNAT --to-source snat_ip
Work breakdown

[ ] new APIs for EgressPolicy
[ ] antrea-controller converts Policies and calculates Groups
[ ] iptables initialization and interface
[ ] openflow initialization and interface
[ ] antrea-agent gets Policies and Groups, and invokes dataplane interfaces

Alternative solutions that you considered
NONE
Test plan
Add E2E tests to verify specific Pods are translated to specific IP when accessting an http server deployed "outside" the cluster (it could be a host-network Pod running on a Node that is different from the Egress Node.
Additional context
Any other relevant information.

@jianjuns I created this issue to track the design change and progress of the first version of the feature, copied some content we have discussed in #667 here. Feel free to update it directly if you have any ideas on details or names.

cc @ceclinux to take a look at work breakdown.

@jianjuns

A question for controlplane API: could we reuse AppliedToGroup instead of adding a new EgressGroup?

I thought about this but didn't find real benefits to do so so switched to another way that could reduce code redundancy and grouping caculation between all kinds of groups, including clustergroups, appliedtogroups, addressgroups and egressgroups.
Some cons of reusing AppliedToGroup I thought of:

When AppliedToGroups for NetworkPolicy and EgressPolicy are mixed in a single API, the AppliedToGroups for one policy will have to wake up another policy's event handler unnecessarily.
The AppliedToGroup for EgressPolicy may use different strategy to dispatch to agents. Egress nodes may need to know all members of a given group, instead of the ones that are running on it (when we want to support noencap mode or the case that SNAT IP is not reachable from non egress nodes).
Currently the AppliedToGroup is coupled with NetworkPolicyController on both controller and agent side, extracting them for another vertical to reuse is more complex than extracting the grouping logic to a separate module and have different group APIs consuming it. In the latter way, the API is business aware while the grouping process is generic.
My PoC of the latter approach is close to finish, and I have verified it could improve the performance of networkpolicy controller greatly as well as reducing code redundancy. I may push the PR for review in 1 or 2 days.

And for AppliedTo why do not have ClusterGroup and Service reference there? Is it for simplification of 1st version?

I copied the struct from your PR. Supporting ClusterGroup and Service reference should be ok, I don't think of big effort introduced by them. Feel free to add them to the design if you think they should be in 1st version.

But from understanding/troubleshooting perspective, it is much better to use a single type, and map a single ClusterGroup to a single AddressGroup or AppliedToGroup.
If we think too much work to refactor NetworkPolicyController, can we still reusing the same AppliedToGroup type, but create another set of AppliedToGroups in this release?
I think better to support ClusterGroup and Service reference too. I can update my PR with my ideas.

I think you mean having another API path but use the same struct, like "/v1alpha1/egressgroups" will get the new set of AppliedToGroup. However, clientset code is generated based on the name of the struct or the "resourceName" tag of the struct. I think it won't work if we use same struct in same API group as the paths in the generated clientset will be exactly same.
And what do you think the first and second problems I mentioned above, especially the second? I think the group for egresspolicy has more difference from AppliedToGroup for NetworkPolicy: it needs to include podIP information and dispatched to all egress nodes, which is more like AddressGroup for Egress Node but AppliedToGroup for non Egress Node.
With these differences,  do you think it's still worth to consider EgressNode as AppliedToGroup?

@jianjuns Given the fact that all agents need to watch all Egresses and there shouldn't be overlapped groups for Egress, I found there is no much value to have a controlplane Egress API as we could just create a EgressGroup with the same name as the Egress resource (just like Service and Endpoint), then use the Egress's name to get its group on agent side to save many code (the controller in antrea-controller can focus on syncing EgressGroup, antrea-agent can leverage Egress Informer), let me know if you have concern on this. This is the code on antrea-controller side: https://github.com/vmware-tanzu/antrea/pull/1965/commits/178405b1eaeee750d8fdb8e4f689165d7523e16d

I am fine to watch Egresses directly from K8s API for now. We can decide what to do later (when we have another solution to discover/assign SNAT IPs).

All code changes have been merged, closing


Ubuntu can't successfully deploy Chinese documents

@0xTim
Hi Tim
After several recent PRs were merged, it was not updated to the website: https://cn.docs.vapor.codes/4.0/deploy/heroku/
I'm not quite sure which step is the issue.
Can you help to check this ./github-actions/deploy-docs-cn.sh script on Ubuntu server?

@BennyDeBock
Hi, Benny
The script seems to have failed to execute, can you confirm it?

@Jinxiansen I can confirm indeed that the script has failed. I'll continue working on it today. For some reason it doesn't work (even if it's mainly the same script as the german documentation).


Show Function Signature/Variable Type in Completion Menu

Describe the solution you'd like
In many LSP-enabled text editors, the function signature or variable type is shown in the completion menu. Consider how this is done in VS Code:

This makes it clear what a particular completion option is/does.

@Kethku Any thoughts on this?

hmm... I've never felt that this was missing, but it might be because this was addressed soon after this issue was open?
either way, function signatures and variable types are correctly shown on the completion menus from what I can see
sorry for the noise, I found this issue while searching for duplicates for another one I was opening, and I thought that it could be closed if it has been addressed

Hey @dnjp, we have added additional information to the autocompletion menu since this issue was created.  Can you check and see if we are providing enough for the languages you prefer?  And also, if we are providing it in a way that is clear to you?  Thanks.

I prefer design in XCode where class, function, property ... classification is made by icon with leading letter. In my opinion it is much more better then classification only using colors.


Small Plugin Cleanup

This is a small clean-up to the plugin types to remove the Kind type parameter that's being used everywhere. It's not actually adding anything useful since Kind will always just be a string at runtime when dealing with it in the Perses app code and so it was making those types a lot more verbose and confusing than they actually needed to be.
I also cleaned up a few other small things and I'll try and leave comments inline to describe what's up. I think I'll have a slightly larger refactoring of those plugin types coming soon that should really clean things up and make them less confusing and verbose.

Sorry about the CI noise if anyone else is getting notified, trying out a few things.


Drop node v8 support, upgrade prettier

closes #763
https://github.com/nodejs/Release/tree/5a579babfec65b0b8354693188ee7a28801a60e1

Travis CI wasn't reporting the status properly. Trying with a different PR.


"content_harvest" fails for entire collection 26762; 1 item (?) has invalid or corrupted image file

Issue: CSU Chico's collection 26762 (CONTENTdm) fails at the "content_harvest" point; from the logs, it looks like it is tripping up on 1 item (?), an image object which appears to have an invalid or corrupted image file. Legacy harvester somehow took the object in, and we're displaying a missing thumbnail. Can we exclude / skip over this object?
Collection ID: 26762
Rikolti mapper type: ETL
Airflow Run ID: manual__2024-05-09T22:06:14+00:00
Airflow log: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/dags/harvest_collection/grid?base_date=manual__2024-05-09T22%3A06%3A14%2B00%3A00&num_runs=365&dag_run_id=manual__2024-05-09T22%3A06%3A14%2B00%3A00
Item:

CSphere prod view: https://calisphere.org/item/98c5a06b8ad73576bc52f54c7dbf492e/
CONTENTdm local view: http://archives.csuchico.edu/digital/collection/coll11/id/17719
Solr: https://harvest-stg.cdlib.org/solr/dc-collection/select?q=coll11%2F17719&wt=json&indent=true
CouchDB ID: 26762--oai:archives.csuchico.edu:coll11/1771
IsShownBy: http://archives.csuchico.edu/utils/getthumbnail/collection/coll11/id/17719

==
[2024-05-09, 15:15:17 PDT] {{taskinstance.py:1824}} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/airflow/dags/rikolti/dags/shared_tasks/content_harvest_operators.py", line 116, in execute
    return super().execute(context)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py", line 76, in wrapper
    return func(*args, session=session, **kwargs)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/providers/amazon/aws/operators/ecs.py", line 476, in execute
    self._start_wait_check_task(context)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/providers/amazon/aws/hooks/base_aws.py", line 743, in decorator_f
    return fun(self, *args, **kwargs)
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/providers/amazon/aws/operators/ecs.py", line 511, in _start_wait_check_task
    self._check_success_task()
  File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/providers/amazon/aws/operators/ecs.py", line 650, in _check_success_task
    raise AirflowException(
airflow.exceptions.AirflowException: This task is not in success state - last 100 logs from Cloudwatch:
Traceback (most recent call last):
  File "/content_harvester/by_record.py", line 153, in get_dimensions
    return Image.open(filepath).size
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/PIL/Image.py", line 3339, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/tmp/11683c25ed8b744e440595fd35262ddf'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/content_harvester/by_page.py", line 98, in <module>
    print_value.append(harvest_page_content(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/content_harvester/by_page.py", line 31, in harvest_page_content
    record_with_content = harvest_record_content(
                          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/content_harvester/by_record.py", line 113, in harvest_record_content
    dimensions = get_dimensions(thumbnail.tmp_filepath, record['calisphere-id'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/content_harvester/by_record.py", line 155, in get_dimensions
    raise Exception(
Exception: PIL.UnidentifiedImageError for calisphere-id oai:archives.csuchico.edu:coll11/17719: cannot identify image file '/tmp/11683c25ed8b744e440595fd35262ddf'
Harvesting content for 100 records at 26762/vernacular_metadata_2024-05-09T22:06:25/mapped_metadata_2024-05-09T22:07:22/data/135.jsonl
Harvested 100 thumbnail records
100/100 described a thumbnail source
Source Thumbnail Mimetypes: Counter({None: 100})
Destination Thumbnail Mimetypes: Counter({'image/jpeg': 100})
Harvesting content for 100 records at 26762/vernacular_metadata_2024-05-09T22:06:25/mapped_metadata_2024-05-09T22:07:22/data/136.jsonl
[2024-05-09, 15:15:17 PDT] {{taskinstance.py:1345}} INFO - Marking task as FAILED. dag_id=harvest_collection, task_id=content_harvesting.content_harvest, map_index=27, execution_date=20240509T220614, start_date=20240509T221314, end_date=20240509T221517
[2024-05-09, 15:15:17 PDT] {{logging_mixin.py:150}} INFO - Message sent to SNS with Message ID: 7b431d36-f024-5728-ac7c-4381bd11e2d8
[2024-05-09, 15:15:17 PDT] {{standard_task_runner.py:104}} ERROR - Failed to execute job 129133 for task content_harvesting.content_harvest (This task is not in success state - last 100 logs from Cloudwatch:
Traceback (most recent call last):
  File "/content_harvester/by_record.py", line 153, in get_dimensions
    return Image.open(filepath).size
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/PIL/Image.py", line 3339, in open
    raise UnidentifiedImageError(msg)
PIL.UnidentifiedImageError: cannot identify image file '/tmp/11683c25ed8b744e440595fd35262ddf'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/content_harvester/by_page.py", line 98, in <module>
    print_value.append(harvest_page_content(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/content_harvester/by_page.py", line 31, in harvest_page_content
    record_with_content = harvest_record_content(
                          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/content_harvester/by_record.py", line 113, in harvest_record_content
    dimensions = get_dimensions(thumbnail.tmp_filepath, record['calisphere-id'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/content_harvester/by_record.py", line 155, in get_dimensions
    raise Exception(
Exception: PIL.UnidentifiedImageError for calisphere-id oai:archives.csuchico.edu:coll11/17719: cannot identify image file '/tmp/11683c25ed8b744e440595fd35262ddf'
Harvesting content for 100 records at 26762/vernacular_metadata_2024-05-09T22:06:25/mapped_metadata_2024-05-09T22:07:22/data/135.jsonl
Harvested 100 thumbnail records
100/100 described a thumbnail source
Source Thumbnail Mimetypes: Counter({None: 100})
Destination Thumbnail Mimetypes: Counter({'image/jpeg': 100})
Harvesting content for 100 records at 26762/vernacular_metadata_2024-05-09T22:06:25/mapped_metadata_2024-05-09T22:07:22/data/136.jsonl; 213)
[2024-05-09, 15:15:17 PDT] {{local_task_job_runner.py:225}} INFO - Task exited with return code 1
[2024-05-09, 15:15:17 PDT] {{taskinstance.py:2653}} INFO - 0 downstream tasks scheduled from follow-on schedule check

I think that once we pull in the the PR for issue 899, the content harvester will just print a warning and we won't have a thumbnail: https://github.com/ucldc/rikolti/pull/917

@aturner I went ahead and merged that PR into main. Can you try re-running the content harvest step for the one failed mapped task?

Thanks, Barbara -- re-running that step now (observing a similar issue with collection 26203, https://github.com/ucldc/rikolti/issues/933 -- will also re-run the step for that one, too!)

@aturner You should be able to just clear and rerun the tasks that failed, btw. No need to rerun the tasks that succeeded!

That did the trick -- success!!! THANKS @barbarahui !!!

Resolved with PR for issue 899


Get Valid Events for current state

Is there a way to get the valid events the current state can take?
Looking at the declaration of fsm.can it looks like it would just be finding all the events that include the current state in from?
My use case is getting all the valid events and creating a button for each valid event.

This was added in 2.3.4 with the transitions method


Move Gradle build from "install" to "script" section of .travis.yml.

The Gradle build command is the command that should determine whether the build
succeeds or fails, so it should go in the "script" section.  Failed commands in
the "install" section mark the build as "Errored".

Codecov Report

Merging #227 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #227   +/-   ##
=========================================
  Coverage     94.85%   94.85%
  Complexity      440      440
=========================================
  Files            62       62
  Lines          1342     1342
  Branches         92       92
=========================================
  Hits           1273     1273
  Misses           42       42
  Partials         27       27

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6fd3b51...267c958. Read the comment docs.

Thanks!


Better integration with Ray Serve

I use several models using Ray Serve outside of Vllm.
Vllm uses all vram, making it impossible to use other models alongside.
So I use the gpu_memory_utilization parameter to limit the memory requested by the worker, which then allows placement_group_bundles to be used in Ray Serve deployments.
The RayWorker class has also been renamed to RayWorkerVllm to avoid ambiguities with other Ray actors.

I think this feature fixes the underlying cause for the problem here: https://github.com/ray-project/ray-llm/issues/94
Thanks @FlorianJoncour!

Yes it seems to be the same issue.
I almost went crazy trying to get this to work before diving into the vllm code

@FlorianJoncour @Yard1 It seems this change causes a bug when gpu_memory_utilzation < 0.5 and tensor_parallel_size > 1:
RuntimeError: CUDA error: invalid device ordinal

I guess this is because num_gpus is set to gpu_memory_utilization? Do you have any idea to fix this?

@WoosukKwon I see. For now we should do:
num_gpus=self.cache_config.gpu_memory_utilization if self.parallel_config. tensor_parallel_size < 2 else 1


Cashed properties for time maps in Part objects

For many use cases, Part objects do not change after being created. Nevertheless, attributes such as beat_map and quarter_map are being computed every time, which can be very slow for large pieces. Therefore, it would be useful to just compute these properties once.

That is true. I tend to use the workaround of assigning the map to a variable beatmap = part.beat_map and reusing this variable. If the maps are cached we'd need to make sure they're always up to date when adding or removing TimedObjects to or from a part which might be a bit messy.


Build errors on Beaglebone Black

Required Info

Camera Model
D435i

Firmware Version
None

Operating System & Version
Ubuntu 16.04

Kernel Version (Linux Only)
4.4.155-ti-r151

Platform
Beaglebone Black

SDK Version
None

Language
None

Segment
Robot

Issue Description
Hi,
I'm trying to build on a Beaglebone Black. I'm following the instructions here and here. When I run make it fails (in this case I'm running with ``VERBOSE=1` flag):
[ 41%] Building CXX object CMakeFiles/realsense2.dir/src/media/playback/playback_sensor.cpp.o
/usr/bin/c++   -DBUILD_EASYLOGGINGPP -DBUILD_STATIC=1 -DELPP_NO_DEFAULT_LOG_FILE -DELPP_THREAD_SAFE -DHWM_OVER_XU -DRS2_USE_V4
L2_BACKEND -DUNICODE -DUSE_SYSTEM_LIBUSB -DWITH_TRACKING=1 -Drealsense2_EXPORTS -isystem /usr/include/libusb-1.0 -I/home/ubunt
u/Downloads/librealsense/realsense2 -I/home/ubuntu/Downloads/librealsense/src -I/home/ubuntu/Downloads/librealsense/third-part
y/realsense-file/rosbag/console_bridge/include -I/home/ubuntu/Downloads/librealsense/third-party/realsense-file/rosbag/cpp_com
mon/include -I/home/ubuntu/Downloads/librealsense/third-party/realsense-file/rosbag/rosbag_storage/include -I/home/ubuntu/Down
loads/librealsense/third-party/realsense-file/rosbag/roscpp_serialization/include -I/home/ubuntu/Downloads/librealsense/third-
party/realsense-file/rosbag/rostime/include -I/home/ubuntu/Downloads/librealsense/third-party/realsense-file/rosbag/roscpp_tra
its/include -I/home/ubuntu/Downloads/librealsense/third-party/realsense-file/rosbag/roslz4/include -I/home/ubuntu/Downloads/li
brealsense/third-party/realsense-file/rosbag/msgs -I/home/ubuntu/Downloads/librealsense/third-party/realsense-file/boost -I/ho
me/ubuntu/Downloads/librealsense/third-party/realsense-file/lz4/lib -I/home/ubuntu/Downloads/librealsense/include -I/home/ubun
tu/Downloads/librealsense/third-party/libtm/libtm/include  -fPIC -pedantic -g -Wno-missing-field-initializers -Wno-switch -Wno
-multichar -Wsequence-point -Wformat-security -mfpu=neon -mfloat-abi=hard -ftree-vectorize -pthread -fPIC   -std=gnu++11 -o CM
akeFiles/realsense2.dir/src/media/playback/playback_sensor.cpp.o -c /home/ubuntu/Downloads/librealsense/src/media/playback/pla
yback_sensor.cpp
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
CMakeFiles/realsense2.dir/build.make:566: recipe for target 'CMakeFiles/realsense2.dir/src/media/playback/playback_sensor.cpp.
o' failed
make[2]: *** [CMakeFiles/realsense2.dir/src/media/playback/playback_sensor.cpp.o] Error 4
make[2]: Leaving directory '/home/ubuntu/Downloads/librealsense/build'
CMakeFiles/Makefile2:68: recipe for target 'CMakeFiles/realsense2.dir/all' failed
make[1]: *** [CMakeFiles/realsense2.dir/all] Error 2
make[1]: Leaving directory '/home/ubuntu/Downloads/librealsense/build'
Makefile:127: recipe for target 'all' failed

Thanks in advance for any help!

Hi @jdwolk
We actually have a note regarding this error in installation.md.
Adding a swap file will solve the problem.

@dorodnic got it, thanks for pointing that out. I was able to build by creating a swapfile using this:
cd /; sudo fallocate -l 1G /swapfile;
sudo chmod 600 /swapfile;
sudo mkswap /swapfile;
sudo swapon /swapfile;
sudo swapon -s;

Edit /etc/fstab, add:
/swapfile   none    swap    sw    0   0

I'm now having problems with realsense-viewer segfaulting + most of the rs- example commands are returning "Frame didn't arrived within 5000". I'll close this issue since I was able to get the build working though.


add quill version 4.3.0

quill is a C++17 low latency logging library.  The project can be found here

Hello @bazelbuild/bcr-maintainers, modules without existing maintainers (quill) have been updated in this PR. Please review the changes.

Please add the presubmit-auto-run and skip-url-stability-check labels to the PR. Thanks!

any advice on how to fix the CI builds please ?

You need to add platforms as a direct dependency, I pushed a new commit to do that.

Can you please update the MODULE.bazel file patch and its integrity value?

thank you @meteorcloudy !

Thanks for the contribution! Do you mind also upstreaming the BUILD and MODULE.bazel file so that we don't need those patches for future version?

Thanks for the contribution! Do you mind also upstreaming the BUILD and MODULE.bazel file so that we don't need those patches for future version?

I am planning to upstream them. I was thinking of placing them within a /bazel directory, following the pattern used by libfmt here.
Does that align with what you had in mind, or would you prefer them at the root directory? I'm flexible either way.

To get Bazel working with {fmt} you can copy the files BUILD.bazel, MODULE.bazel, WORKSPACE.bazel, and .bazelversion from this folder (support/bazel) to the root folder of this project.

If it's in a subfolder, users still need to copy it out to make the Bazel build work, which means BCR module still needs a patch file. So if you don't mind, it's better to add those files where Bazel expects them to be.


Druid ingestion ends with SUCCESS status but exception is thrown at the end

Hi guys,
I run a ingestion job druid (<IP_ADDRESS>) + hadoop 2.7.1 (custom version declared in common conf druid.extensions.hadoopDependenciesDir=/druid/hadoop-dependencies).
As deep storage - hdfs in a docker container.
Loaded extensions: druid.extensions.loadList=["druid-histogram", "druid-datasketches", "postgresql-metadata-storage", "druid-kafka-indexing-service", "druid-hdfs-storage"]
The job finishes successfully, data is ingested but at the very end i get the next exception:

2017-01-13T09:44:48,064 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
"id" : "index_hadoop_impression_segments5_2017-01-13T09:42:05.410Z",
"status" : "SUCCESS",
"duration" : 159082
}

after fiew lines:

2017-01-13 09:44:48,136 Thread-2 ERROR Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Not started
at io.druid.common.config.Log4jShutdown.addShutdownCallback(Log4jShutdown.java:45)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:273)
at org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:284)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:273)
at org.apache.hadoop.hdfs.LeaseRenewer.(LeaseRenewer.java:72)
at org.apache.hadoop.hdfs.DFSClient.getLeaseRenewer(DFSClient.java:699)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:859)
at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:853)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2407)
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2424)
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)

Original report in https://groups.google.com/d/topic/druid-user/5qKGnXsmqMw/discussion. I believe this is harmless, but annoying.

In my case job is failing. No other error is there in the log. Could you please confirm if is failing not because of this issue?

@gianm hi gianm, I am stuck with this issue too. and the historical node wasn't aware of the indexing task at all. so definitely there is no segment file generated. That is not so "harmless" in our case...
Do you know what the reason is causing this problem? I can't find any other error log message to debug. Great appreciation if there is any workaround to avoid this issue. Thanks!

i meet this to

I use  imply-2.0.0 and hadoop 2.7.3
my log is
2017-10-16T07:05:17,572 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
  "id" : "index_hadoop_apmbox-data_2017-10-16T06:45:53.604Z",
  "status" : "SUCCESS",
  "duration" : 1148556
}
2017-10-16T07:05:17,583 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.server.coordination.AbstractDataSegmentAnnouncer.stop()] on object[io.druid.server.coordination.BatchDataSegmentAnnouncer@539c4830].
2017-10-16T07:05:17,583 INFO [main] io.druid.server.coordination.AbstractDataSegmentAnnouncer - Stopping class io.druid.server.coordination.BatchDataSegmentAnnouncer with config[io.druid.server.initialization.ZkPathsConfig@22e2266d]
2017-10-16T07:05:17,583 INFO [main] io.druid.curator.announcement.Announcer - unannouncing [/druid/announcements/druidio.cdn.gz4399.com:8109]
2017-10-16T07:05:17,604 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.server.listener.announcer.ListenerResourceAnnouncer.stop()] on object[io.druid.query.lookup.LookupResourceListenerAnnouncer@7994a0d1].
2017-10-16T07:05:17,604 INFO [main] io.druid.curator.announcement.Announcer - unannouncing [/druid/listeners/lookups/__default/druidio.cdn.gz4399.com:8109]
2017-10-16T07:05:17,606 INFO [main] io.druid.server.listener.announcer.ListenerResourceAnnouncer - Unannouncing start time on [/druid/listeners/lookups/__default/druidio.cdn.gz4399.com:8109]
2017-10-16T07:05:17,606 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.query.lookup.LookupReferencesManager.stop()] on object[io.druid.query.lookup.LookupReferencesManager@7a66c35a].
2017-10-16T07:05:17,606 INFO [main] io.druid.query.lookup.LookupReferencesManager - Stopping lookup factory references manager
2017-10-16T07:05:17,612 INFO [main] org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@3457cc8d{HTTP/1.1}{<IP_ADDRESS>:8109}
2017-10-16T07:05:17,615 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.s.ServletContextHandler@1785d194{/,null,UNAVAILABLE}
2017-10-16T07:05:17,618 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.indexing.worker.executor.ExecutorLifecycle.stop() throws java.lang.Exception] on object[io.druid.indexing.worker.executor.ExecutorLifecycle@73afe2b7].
2017-10-16T07:05:17,619 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.indexing.overlord.ThreadPoolTaskRunner.stop()] on object[io.druid.indexing.overlord.ThreadPoolTaskRunner@1c84d80a].
2017-10-16T07:05:17,620 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.discovery.ServerDiscoverySelector.stop() throws java.io.IOException] on object[io.druid.curator.discovery.ServerDiscoverySelector@285583d4].
2017-10-16T07:05:17,624 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.announcement.Announcer.stop()] on object[io.druid.curator.announcement.Announcer@9accff0].
2017-10-16T07:05:17,625 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.curator.discovery.ServerDiscoverySelector.stop() throws java.io.IOException] on object[io.druid.curator.discovery.ServerDiscoverySelector@83bb0f].
2017-10-16T07:05:17,625 INFO [main] io.druid.curator.CuratorModule - Stopping Curator
2017-10-16T07:05:17,626 INFO [Curator-Framework-0] org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
2017-10-16T07:05:17,628 INFO [main] org.apache.zookeeper.ZooKeeper - Session: 0x15f22e66fb00049 closed
2017-10-16T07:05:17,628 INFO [main-EventThread] org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x15f22e66fb00049
2017-10-16T07:05:17,628 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.http.client.NettyHttpClient.stop()] on object[com.metamx.http.client.NettyHttpClient@5e26f1ed].
2017-10-16T07:05:17,648 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.storage.hdfs.HdfsStorageAuthentication.stop()] on object[io.druid.storage.hdfs.HdfsStorageAuthentication@4c51077d].
2017-10-16T07:05:17,648 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.metrics.MonitorScheduler.stop()] on object[com.metamx.metrics.MonitorScheduler@733c464f].
2017-10-16T07:05:17,648 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void com.metamx.emitter.service.ServiceEmitter.close() throws java.io.IOException] on object[com.metamx.emitter.service.ServiceEmitter@180b3819].
2017-10-16T07:05:17,648 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking stop method[public void io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner.stop()] on object[io.druid.initialization.Log4jShutterDownerModule$Log4jShutterDowner@3f6f3cc].
2017-10-16 07:05:17,690 Thread-2 ERROR Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Not started
	at io.druid.common.config.Log4jShutdown.addShutdownCallback(Log4jShutdown.java:45)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:273)
	at org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256)
	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)
	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182)
	at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103)
	at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
	at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42)
	at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:284)
	at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
	at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
	at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
	at org.apache.hadoop.hdfs.LeaseRenewer.<clinit>(LeaseRenewer.java:72)
	at org.apache.hadoop.hdfs.DFSClient.getLeaseRenewer(DFSClient.java:699)
	at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:859)
	at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:853)
	at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2407)
	at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2424)
	at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)

Any progress？

I saw the same issues from overlord's log via overlord console GUI page. Initially I also think this is a bug. But after survey, I found that my historical node is down as I forget to bring this node up. After historical node is up, so I can query the result from broker now and druid system behave well.

BTW, I use druid 0.12.0 which carries extension of  hadoop 2.7.3, my Hadoop cluster uses hadoop 2.9.0.
So if this is case in your environment, pay attention to different hadoop version issue. See
http://druid.io/docs/latest/operations/other-hadoop.html

I have encountered the same error，the druid version is 0.12.2.
How to solve it?
Can anyone give me a suggestion?

This exception 2017-10-16 07:05:17,690 Thread-2 ERROR Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Not started is safe to ignore.
If a task is failing, it is failing for some other reason.

I also  meet this issue
2018-10-29 09:35:15,235 Thread-3 ERROR Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Not started 	at io.druid.common.config.Log4jShutdown.addShutdownCallback(Log4jShutdown.java:47) 	at org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:273) 	at org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256) 	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216) 	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:145) 	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41) 	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:182) 	at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:103) 	at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43) 	at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:42) 	at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29) 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:253) 	at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155) 	at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132) 	at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685) 	at org.apache.hadoop.hdfs.LeaseRenewer.<clinit>(LeaseRenewer.java:72) 	at org.apache.hadoop.hdfs.DFSClient.getLeaseRenewer(DFSClient.java:818) 	at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:956) 	at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:1079) 	at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2760) 	at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2777) 	at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)

https://groups.google.com/forum/m/?utm_medium=email&utm_source=footer#!msg/druid-user/5qmrmMEDD8g/yrh7wOVVBAAJ
I just posted about my similar issue in the Druid forums at the link above. I would be very grateful if you could recommend how to fix my problem. I am also getting the unable to register shutdown hook in my logs. Thank you very much

I have same problem .how to solve it?

my druid.version is 0.12.3

Hi Team.. I am also facing the same error and in my case, no data is ingested into segment. My segment granularity is hour, so after a certain point, no data is getting ingested into druid and it's throwing this error only. The task status is SUCCESS. Not sure how to debug it. It would be great help if you can help with it.


Delayed restart after suspend/sleep

I have a Bluetooth speaker as a slave in a Master/Slave power strip with the main system as a Master.
This helps enormously with energy savings.
When the system wakes up, the service needs to restart with delay so the just powered on Bluetooth speaker can reconnect to the system.
One way to solve the problem is by creating an executable reconnect_to_bluetooth.sh at /lib/systemd/system-sleep:
#!/bin/sh
case $1/$2 in
  pre/*)
    echo "Other"
    ;;
  post/*)
    sleep 40 # Because Bluetooth speaker turns on in about 30 seconds
    systemctl restart bluetooth-autoconnect.service
    ;;
esac

Just opened it down for future reference.

Restarting the daemon is a bit overkill. A one off bluetooth-autoconnect should also work.


Future Integration with Turbopack

It looks like Vercel will move to Turbopack https://vercel.com/blog/turbopack in the future.
Do you have any plans to speak with Vercel to have the support of Module Federation with Turbopack?
And I think once they migrate to Turbopack; they will deprecate the Webpack option in the next config

Ive already DM'd tobias regarding this. Turbo is heavily inspired off webpack, it looks almost the same - the intent of turbo is to provide an API on par with webpacks, so there's a very high likelihood that we will be able to support it natively
If not, we still have ways to solve this - all in all I'm not concerned, but turbo is way to alpha to try integrating this tech as of yet.
Its still a long ways off from being useable

Webpack will also remain supported, its not a abandonment. Just another option alongside webpack

@ScriptedAlchemy Sounds pretty optimistic:) Thanks for answering.


Workshop: freeCodeCamp - HTML5 and CSS

https://github.com/nesterone/sf-eng-init-gl-spring-2016/blob/master/02-browser/02-basic-html-css-jquery/basic-html-css-jquery.md#workshop-freecodecamp---html5-and-css

C:\Users\daria.bieloborodova\Desktop\w1css.png


The part of Conditional Image Generation and Training

Thank you for your excellent work and I would like to ask when the part of Conditional Image Generation and Training will be made public?

Maybe we can edit a code to use the conditional image generation, you can see the code files and there are some classes for conditional image generation I think.

Closing this issue as I will be posting updates to this issue: https://github.com/samar-khanna/DiffusionSat/issues/1


fix: avoid starving background task permits in eviction task

As seen with a recent incident, eviction tasks can cause pageserver-wide permit starvation on the background task semaphore when synthetic size calculation takes a long time. Metric links can be found in the internal slack thread.
As a solution, release the permit while waiting for the state guarding the synthetic size calculation. This will most likely hurt the eviction task eviction performance, but that does not matter because we are hoping to get away from it using OnlyImitiate policy anyway and rely solely on disk usage-based eviction.

Hotfix to be included in #7447 -- please review accordingly. Rationale: we currently have at least 2h long starvations.

Realized that if all compaction tasks are blocked, we will not have any early flushes due to memory pressure either.


Default the selected class to the first class in the list

@jeroenooms - Current version doesn't have Hongsuda's feedback "at least one class will need to be selected.. So default to the first class in the list"

We decided against this to force the user to select a class.

@jeroenooms - Thanks for the update! I'm closing this then.


Better handling of errors when processing messages

Errors are common when processing message content. Currently if a message is rejected we may end up with a lot of messages with higher sequence numbers stuck in the message buffer. The messages will also keep being replicated over and over again via various means even though we can't consume them.
Overall it is a bit unclear what to do if we know we can't handle a specific message. Perhaps some kind of a back-off mechanism is needed.

I think we can consider this fixed in 3cc924d98ba1f6a6cf4d9a6d424db46e1ba6ee3a and 5a5d6404665c760bd8cc1cd3ec0357a0f746c520.


Fix namespace of cross-db macros from dbt_utils to dbt

Are you a current Fivetran customer?

What change(s) does this PR introduce?

Since dbt_utils 1.0.0 the cross-db macros are in the dbt namespace https://github.com/dbt-labs/dbt-utils/releases/tag/1.0.0.
Did you update the CHANGELOG?

[ ] Yes

Does this PR introduce a breaking change?

[ ] Yes (please provide breaking change details below.)
[ ] No  (please provide an explanation as to how the change is non-breaking below.)

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

[ ] Yes

Is this PR in response to a previously created Bug or Feature Request

[ ] Yes, Issue/Feature [link bug/feature number here]
[ ] No

How did you test the PR changes?

[ ] Buildkite
[ ] Local (please provide additional testing details below)

Select which warehouse(s) were used to test the PR

[ ] BigQuery
[ ] Redshift
[ ] Snowflake
[ ] Postgres
[ ] Databricks
[ ] Other (provide details below)

Provide an emoji that best describes your current mood

:dancer:
Feedback
We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your feedback on our existing dbt packages or what you'd like to see next.

Oh man I came here to make this PR and someone beat me to it! 😁 Commenting to follow along since this is blocking our upgrade to the latest versions of everything.

Thanks for the note @erika-e and raising that this is blocking other users. As that is the case, I may elect to make the required changes directly in order to prioritize getting this fix out the door.

@fivetran-joemarkiewicz Thanks for taking care of this! I was out for a long weekend so it's good that you went ahead and did the update :)


Only run docker-disk-gc on low-disk

This riffs on #325 and supersedes #317. It adds a disk free check to the cron script that cleans up docker things so that older images/containers/volumes are only purged if disk or inodes are low.
It also increases the frequency of this to hourly (cron.hourly). I was almost tempted to run this in the environment hook too, so that issues are surfaced during the build if possible.
Feels @patrobinson?

Yup, agreed @patrobinson. I've made those changes.

The other thing this does is marks the instance as unhealthy if it has low disk and an image purge won't fix it.

Sounds pretty sweet. I like it! 👍


pyswagger with javax.json.JsonObject valuetypes NUMBER & STRING

Hi,
Could you help with one issue (if it's an issue): I'm using pyswagger library for rest api testing based on swagger.json file, everything worked absolutely fine until the moment when we tried the reference for
JsonObject (javax.json) in swagger.json file:
..."$ref" : "#/definitions/JsonValue"
"definitions" : {
    "JsonValue" : {
      "type" : "object",
      "properties" : {
        "valueType" : {
          "type" : "string",
          "enum" : [ "ARRAY", "OBJECT", "STRING", "NUMBER", "TRUE", "FALSE", "NULL" ]
        }
      }

now by pushing usual json data: {'key': 'value'}, library returns always:
...  File "C:\Python27\lib\site-packages\six.py", line 599, in iteritems
return d.items(**kw)
AttributeError: 'str' object has no attribute 'items'
By pushing same json-data with Postman (and other tools) and swagger UI it works perfectly fine, and valueType is interpreted as STRING in our rest API endpoint.
Any chance you could help, what could be the issue? Any tricks?
Thanks in advance.
Alexey

I'm not very clear about this issue you described, did you mean:

you have an API, which accept JsonValue as a parameter
you provide a string, and wish this library to convert it to JsonValue.

Could you provide a code example about "how to call that API"?

Thanks for quick reply, here are more details, hopefully it will make it more clear:
We have a Java EE 7 JAX-RS rest endpoint that accepts javax.json.JsonObject as a PUT method parameter. One javax.json.JsonObject contains N amount of javax.json.JsonValue objects which all have a key as a String.
JsonObject structure:
{
key1:{jsonvalue1},
key2:{jsonvalue2},
}
For example, CURL command that works:
curl -H "Content-Type: application/json" -X PUT -i -k -d '{"keyToSomeValue": 5}' http://:/
So the data is really simple json object containing key value pair. For example in this case the ValueType of this JsonValue would be ValueType.NUMBER and value of that JsonValue would be 5. And the key is keyToSomeValue.
JsonObject structure:
{
keyToSomeValue: {5} ,
}
curl -H "Content-Type: application/json" -X PUT -i -k -d '{"keyToSomeValue2": “other value”}' http://:/
JsonObject structure:
{
keyToSomeValue2: {“other value”} ,
}
Now our problem is that we are not able to send this kind of json data using pyswagger library. JsonValue ValueType is always interpreted as ValueType.OBJECT because we are not able to send simple {‘key’:value} data. We are only able to send for example key={json object}.
Thanks!

We have found an issue on our side. The issue was in usage of:
"schema" : {
"type" : "object",
"additionalProperties" : {
"$ref" : "#/definitions/JsonValue"
}
"If additionalProperties is an object, that object is a schema that will be used to validate any additional properties not listed in properties." This was the problem in our case.
Thanks a lot for your support and providing this library. And sorry for false alarm.
We can close this item.

Good to know that your problem is resolved, and it's ok for false alarm.


fix: server actions by removing origin group + distribution refactor

Fixes #129

Change default behavior from OriginGroup so that POST methods can be used to enable server actions
Refactor massive createDistribution so more readable/maintainable
Change variables referencing “lambda” to “server” within distribution. “Lambda” is confusing b/c there are multiple lambdas (image opt + revalidation). Server is more descriptive.
Use cloudfront.CachePolicy.CACHING_OPTIMIZED for static behavior. SST's NextjsSite has this. Simplifies our code, too.
Updated cached headers for server behavior following SST.
Rename NextjsLambda to NextjsServer and ImageOptimizationLambda to NextjsImage
Remove handleS3Request code from Lambda@Edge
Fix image requests when function url auth type is IAM.

Questions

Before, the lambda (now server) behavior had cachePolicy: cloudfront.CachePolicy.CACHING_DISABLED. I've updated it to use the NextjsDistribution.serverCachePolicyProps which can be configured by user. Previously, this cache policy was only being used by the default behavior (origin group with s3 + fallback to server) which is a little confusing to me. Should I keep the disabled cache but allow user to configure? I don't think I should.

@revmischa, was the managed origin request policy, AllViewerExceptHostHeader, documented on this page ever considered to fix the host issue for CloudFront - Lambda Function URLs instead of Lambda@Edge?

@revmischa, was the managed origin request policy, AllViewerExceptHostHeader, documented on this page ever considered to fix the host issue for CloudFront - Lambda Function URLs instead of Lambda@Edge?

Hm I didn't know about it, give it a try! Good find.

That policy is relatively new. NextjsSite also uses it!

That policy is relatively new. NextjsSite also uses it!

@khuezy, was NextjsSite able to ditch Lambda@Edge by using it?

I think so, NextjsSite only references the EdgeFunction for "edge" mode.

Updates:

Remove Lambda@Edge for lambda function urls with no auth. Still need Lambda@Edge to sign request for function urls with IAM auth.
Update the origin request policy of image behavior to cloudfront.OriginRequestPolicy.ALL_VIEWER because I couldn't get lambda function url auth type IAM_AUTH to work without it. I don't see any harm in including additional headers/cookies/querystrings that user sends to image optimization function as Next.js will only use needed values (Accept header and url, q, w querystrings)
Update the image cache policy to include all cookies. IMO this is a safer default. What if user's Next.js site runs middleware on _next/image requests to authorize the user to view the image and needs cookies? With current default setup, unauthorized users could see cached images. The downside is a cache key that includes more information, increasing the chance of a cache miss, but I think it's worth it. Users who have more performance concerns can override this default behavior.

Would really appreciate feedback on these updates! Once I receive feedback, I will merge as this PR is ready to go IMO.

Updates:

Remove Lambda@Edge for lambda function urls with no auth. Still need Lambda@Edge to sign request for function urls with IAM auth.
Update the origin request policy of image behavior to cloudfront.OriginRequestPolicy.ALL_VIEWER because I couldn't get lambda function url auth type IAM_AUTH to work without it. I don't see any harm in including additional headers/cookies/querystrings that user sends to image optimization function as Next.js will only use needed values (Accept header and url, q, w querystrings)
Update the image cache policy to include all cookies. IMO this is a safer default. What if user's Next.js site runs middleware on _next/image requests to authorize the user to view the image and needs cookies? With current default setup, unauthorized users could see cached images. The downside is a cache key that includes more information, increasing the chance of a cache miss, but I think it's worth it. Users who have more performance concerns can override this default behavior.

Would really appreciate feedback on these updates! Once I receive feedback, I will merge as this PR is ready to go IMO.

All sounds great to me

Ditto, I agree the .all() is the lesser evil for defaults. I would put a note in the readme about cookie caches b/c I bet there will be people who are on the other end where they're confused why none of the caching works b/c google analytics or something else is injecting unique cookies per request.


Implement series.cumsum() in new style

@shssf, @kozlov-alexey could you take the next round to review the PR?


I  added my name to guestbook.md

Hi,
I just finished the github forking lesson. Thanks for the great course.

Thank you! I am so happy you enjoyed the course!


Export useGtag function

Describe the feature
Currently, we're importing the useGtag function as follows:
import { useGtag } from '#imports';

As we put all our components in Storybook as well, and Storybook can't really cope with #imports (it errors out), I was wondering if you could make the useGtag function available through the package exports itself?
Additional information

[ ] Would you be willing to help implement this feature?
[ ] Can you think of other implementations of this feature?

Final checks

[X] Check existing issues.

Yes, I'm using the @nuxtjs/storybook module. The error is the following:

Would it be a solution to provide the Nuxt context to useGtag as an argument?

I don't know how the Nuxt context is set up within Storybook. Do other auto-imports work? Can you remove the explicit import from #imports and rely on auto-imports instead?

Thanks for your quick replies!
I have been able to bypass all other #imports by using a "direct import". This is the only package left 😃
I have removed the import line, but the issue remains.


Create MIT license.txt

Howdy! I was poking around online for a few quaternion-based IMU examples and came across yours. I noticed that even though it was published in the Arduino library manager (so folks are using it) it doesn't currently state an explicit license. While not a big issue, this is technically problematic under most copyright law.
I went ahead and added an MIT license to start, as it's pretty permissive and I think matches the intended usage of the library? No worries if this isn't desirable at the moment.

Hi Chris,
Thanks for making me of aware of this.. I had no clue it would be an issue.
The license looks fine so I've accepted the pull. :)


Allow createClient() to use user's jwt without the need for anon key

Feature request
Is your feature request related to a problem? Please describe.
Allow createClient() to use user's jwt so that the client does not need anon key:
const supabase = createClient(url, user's jwt);
Currently this will not work for Realtime which gives repeated WebSocket connection failed errors.
Just to provide more context, my web client gets the user's jwt by signing in indirectly through my own custom API server, so it does not need the anon key to call Supabase API.
Describe the solution you'd like
Allow createClient() to use user's jwt without the need for anon key.
Describe alternatives you've considered
Currently the solution is to use anon key for createClient(), followed by:
supabase.auth.setAuth(user's jwt)

Hey @kk21, this was intentional in v1. You can think of the anon key more as an API key rather than a substitute for a user's JWT. We don't plan on changing the behavior to keep it consistent across the other Supabase JS client libraries.


Using an observer to handle menu updates to recalculate the dividers when change while is open

Summary
Using an observer to handle menu updates to recalculate the dividers when
anything inside the menu changes while is open
Ticket Link
MM-14490

@jespino any way you can add tests to this?

@enahum let me check

@enahum is quite hard to test it without mocking the MutationObserver, and if I mock the mutation observer makes no sense to test it.

@thekiiingbob any way we can add e2e tests for this?

@jespino looks like the build is failing

@enahum Yes now the tests are fixed. and about the e2e tests, is doable, the problem is this happen if somebody change the permissions while the menu is open it is not trivial to reproduce. I'll give it a try.

@enahum cypress test added! :tada:

@saturninoabril I added you hear because of the cypress test to be sure everything looks right :)

Awesome awesome awesome that you added a cypress test!

@thekiiingbob yes, was thanks to the "pressure" from @enahum but I'm really glad to have people pushing me in the right direction time to time :)

You are all welcome :p

Creating a new SpinWick test server using Mattermost Cloud.

Mattermost test server created! :tada:
Access here: https://mattermost-webapp-pr-3527.test.mattermost.cloud

Account Type
Username
Password

Admin
sysadmin
Sys@dmin123

User
user-1
User-1@123

Test server destroyed

@bradjcoughlin Please prioritize reviewing this.

@bradjcoughlin Please prioritize reviewing this.

@amyblais I saw two reviewers had approved, wondering if my review is superfluous? If not, happy to review as well.

@bradjcoughlin I think Saturn's review was a QA review to check that the cypress tests are working fine.

@jespino I haven't worked much with MutationObserver() , but abstractly speaking, this makes a lot of sense. I could do some experimentation with it myself, or, if you know of another dev who has experience with this, you could also assign them to review.

@bradjcoughlin I think is ok, is working locally and in cypress.


fix (samples) Import user events GCS: given gsc input path was not found

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[ ] Make sure to open an issue as a bug/issue before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
[ ] Ensure the tests and linter pass
[ ] Code coverage does not decrease (if any source code was changed)
[ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕
If you write sample code, please follow the samples format.

This PR is cloning changes made in #481 , needs to be closed.


[4.0.5] macOS Won't Stop Requesting Accessibility Permission

Hi,
I just installed kwm using Homebrew and immediately the dialogue appeared requesting that I grant access to kwm to use the accessibility features. I followed the link to the Security & Privacy pane within System Preferences and ticked the checkbox next to 'kwm' in Privacy -> Accessibility. However, it's still prompting me every 30 or so seconds to grant permissions.
I've tried reinstalling, rebooting, restarting via 'brew services restart kwm', removing 'kwm' from the list under Privacy -> Accessibility and adding it in manually, nothing seems to work.
Apologies if this has been solved already, I tried searching and couldn't see anything that could help me. If you need any additional information please let me know!

Same issue here. running homebrew install, osx 10.11.3. can't figure it out for the life of me.

Same here. It's really annoying.

Me too. Where is the logfile for the app?

This seems to be a problem related to how brew utilizes the services system (launchd).
If you start kwm using brew services, and then get prompted for accessiblity -> accepts,
you need to restart the brew services for the settings to take effect.


How to iterate chat.say()

Hi! This is a question more than an issue, but I was wondering how to iterate chat.say() while guaranteeing that it runs in the right order (i.e. synchronously?)
Clearly the following doesn't work:
for (var i=0;i<arr.length;i++) {
    chat.say(`${arr[i].something}`)
}

By using then()

@Jimlnth  thanks for the suggestion, but then doesn't help it run synchronously.
I also tried chat.say().then(()=>{return})

@Jimlnth  thanks for the suggestion, but then doesn't help it run synchronously.
I also tried chat.say().then(()=>{return})

Maybe I didn't understand correctly your problem but I'm personally using this way to chain messages synchronously and it always worked like a charm.
chat.say('first message').then(() => {
chat.say('second message');
});

Oh I see what you mean, yeah I've used then() to chain for specific numbers of messages, but I'd like to dynamically choose the number of messages in this case, i.e. via a loop.
Hope you remember what you did!

Hey @danielnjoo, chat.say() returns a promise, so maybe you can try doing something like this:
let promise = chat.say('first message')
for (let i = 0; i < 10; i += 1) {
  promise = promise.then(() => chat.say(`message #${i}`))
}

I'm not 100% sure that would work, but it could take you in the right direction.

This works thanks!


ZEPPELIN-3325: R interpreter build bugfix

What is this PR for?
Failing to build R interpreter (-Pr) after Spark refactoring
What type of PR is it?
Bug Fix
What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-3325

How should this be tested?

mvn clean package
--batch-mode package
-Phelium-dev
-Pscala-2.11
-Dscala.version=2.11.8
-Dscala.binary.version=2.11
-Pbuild-distr
-Pspark-2.1
-Dspark.version=2.1.1
-Pr
-Phadoop-2.7
-Dhadoop.version=2.7.3
-Dmaven.findbugs.enable=false
-Drat.skip=true
-Dcheckstyle.skip=true
-DskipTests

Questions:

Does the licenses files need update? N
Is there breaking changes for older versions? N
Does this needs documentation? N

Thanks for contribution @necosta ， Could you update the jira title ？And also update the .travis.yml seems travis didn't cover r interpreter

Ok, let me update on the issue.
There was a confusion on my side between SparkR and R
Latest master, ff4c0f3551df1, running:
mvn clean package --batch-mode package -Phelium-dev -Pscala-2.11 -Dscala.version=2.11.8 -Dscala.binary.version=2.11 -Pbuild-distr -Pspark-2.1 -Dspark.version=2.1.1 -Pr  -Dhadoop.version=2.7.3 -Dmaven.findbugs.enable=false -Drat.skip=true -Dcheckstyle.skip=true -DskipTests
Error:
[INFO] Zeppelin: R Interpreter ............................ FAILURE [  1.221 s] [INFO] Zeppelin: Helium development interpreter ........... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 23:39 min [INFO] Finished at: 2018-04-13T09:31:24+01:00 [INFO] Final Memory: 372M/1979M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project zeppelin-zrinterpreter_2.10: Could not resolve dependencies for project org.apache.zeppelin:zeppelin-zrinterpreter_2.10:jar:0.9.0-SNAPSHOT: Could not find artifact org.apache.zeppelin:zeppelin-spark_2.11:jar:0.9.0-SNAPSHOT in apache.snapshots (http://repository.apache.org/snapshots) -> [Help 1]
I applied my bugfix on top of latest master (also a code format correction on JDBC) and got a successful build with the same parameters.
@zjffdu , another review? Thanks

ping @zjffdu
https://travis-ci.org/nokia/zeppelin/builds/368034675 (stage 4 looks broken)
Thanks


Not sure about an update

The "master" file or whatever keeps saying it's getting updated, but I can't use that it seems.
However the most current release of the application (0.30.3) keeps telling me there's an update and I just don't see it. I even links back  to it's own download from September.

The most current release was 0.30.4.
Not sure what you mean by "master" file?

The main page with all the file names or whatever, when I click "Clone Download" it calls it a master file.
Where can I download 0.30.4 because I can't seem to see it. Just version 0.30.3, and the update for my version just brings me back to 0.30.3.

Nevermind it suddenly started working for some reason after closing the old TiTS editor and restarting my browser.

Hmm, now that I've downloaded 0.30.4 its saying that there are some unexpected payloads after extracting. Gives me a thing saying "Header:" for both TitsEd.data.xml and TitsEd.exe and they aren't extractable, so it's non functioning.

I replaced the release zip.
I think I know what you were referring to with "master" file.
That link is a zip of the source code of the master branch, so it calls it master.zip.
Unless you plan on compiling it yourself that isn't useful to you.


installation fails on Ubuntu 18.04 due do dependency on ancient json vesion

What went wrong?
Installation failed during test install of 18.04 Ubuntu
see https://gist.github.com/matkoniecz/3e23d7972f9e8d73acbe2678fff9d264
it appears that it depends on json version from 2015 - see https://rubygems.org/gems/json/versions/1.8.2
What steps did you follow?

sudo gem install backup
sudo gem install backup --version 4.3.0 (as it was in template of issue)

How is your copy of backup configured?
not relevant
Tell us about the computer that runs the backup gem
uname -a gives Linux golem 4.18.0-25-generic #26~18.04.1-Ubuntu SMP Thu Jun 27 07:28:31 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Operating system: Lubuntu 18.04
Ruby version: 2.5.1p57
Tools or services used by backup: None

I think it is because of json 1.8.2 is incompatibile with ruby 2.4.0
See this issue https://github.com/flori/json/issues/303
I encounter the same problem. I do not know how to fix it yet.

https://github.com/backup/backup/issues/877
TLDR;
gem install backup -v5.0.0.beta.1

Closing as technically it is solved - though this version will be never released as this project is dead.
Maybe I will submit this unreleased version as a separate gem, and potentially maintain parts that I use?

I am tempted to open a new issue "Release 5.0.0"


yarn conditional may be unreachable in hook template.

The hook template currently contains this logic:
  elif npx @arkweid/lefthook -h >/dev/null 2>&1
  then
    npx @arkweid/lefthook $@
  elif yarn lefthook -h >/dev/null 2>&1
  then
    yarn lefthook $@

However, I'm not sure it's possible for the yarn branch to ever execute. I think, if yarn is installed, npx will almost certainly be installed. If npx is available on the machine, it will automatically download and run @arkweid/lefthook.
I think these branches should be swapped. The yarn branch should only work if lefthook is already installed locally. If it's not already available locally, falling back to npx seems like a good option.
Alternatively, npx has a --no-install flag that will prevent it from automatically installing lefthook.

Got a PR up with the suggested change, hopefully this helps.

Now in master, going to release it in v0.8.0


New twitter UI makes the extension not working

The new Twitter UI makes the extension not working. I had to go back to the previous Twitter UI.

OK, this seems like I need to push new versions ASAP.


Long urls in connections list collide with button

Description
As per title
Steps to Reproduce

Connect with an app with a long url
Open wallet and view connections

Expected behavior
A different layout or simply text truncation
Screenshots

they don't collide anymore, the width of the container expands and becomes scrollable. tried to make it that so it uses ellipsis, but it just seems like a lot more hacky to do it that way. anyway, confirmed that the problem is gone.


[preset-wind] Media breakpoint prefix ignored when using prefix config

UnoCSS version
0.57.2
Describe the bug
With the reproduction below, when running the command:
./node_modules/.bin/unocss --preflights false sample.html
the output uno.css should be:
.tw\:my-1{margin-top:0.25rem;margin-bottom:0.25rem;}
@media (min-width: 768px){
.tw\:md\:my-1{margin-top:0.25rem;margin-bottom:0.25rem;}
}

but the actual output is:
.tw\:my-1{margin-top:0.25rem;margin-bottom:0.25rem;}

Reproduction
https://gist.github.com/kaorukobo/416eccfaddeb654b3325c837a81612be
and also can be reproduced by the following commands:
npm i unocss

cat > sample.html < uno.config.ts <<TS
import { defineConfig } from 'unocss'
import presetWind from '@unocss/preset-wind'

export default defineConfig({
    presets: [
        presetWind({prefix: 'tw:'}),
    ],
    preflights: [],
})
TS

./node_modules/.bin/unocss --preflights false sample.html

cat *.css

System Info
$ node -v
v20.8.0

Validations

[X] Read the Contributing Guidelines.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Use md:tw:my-1. You should make the variant before rules.

@zyyv ah... what a mistake. Thank you.


Class 30 hash tables

[x] verify project, git branch, and file names
[x] create HashTable data structure

[x] implement required methods
[x] test data structure
[x] document data structure

[ ] stretch goals

[ ] define interface for individual bucket class
[ ] implement remove all collisions

Coverage increased (+3.6%) to 94.428% when pulling 33310019ae5402086f817744ebaa28f7bc8ed456 on class-30-hash-tables into 8c75048b545bbc3b82eb0b6b5c18f23426a6836c on master.


[BUG] Search page rolls back to initial state whenever a new search filter is added/removed.

Has this bug been raised before?

[X] I have checked "open" AND "closed" issues and this is not a duplicate

Where did you find this bug?
Production
Version of BioDrop (for example "v1.2.3")
v2.84.4
Description
On the search page, when multiple filters are selected whenever a new filter is added/removed the display data rolls back to the initial state for a moment and then displays the new data.
For example if I have selected react, then the result will be shown for react then if i also select python then for a few seconds i will see the initial unfiltered search result and then the result for react+python filter
Screenshots
No response
Do you want to work on this issue?
Yes
If "yes" to above, please explain how you would technically implement this
No response

Confirmed this bug is happening and I will unlock, however, I am not able to assign this issue to you as you have not explained your approach to solve this.  Can you please add more detail on what you would like to do to address the issue?

I would like to work on this issue , can you assign this to me . I know I have another issue, but I solved it and  sent a pull request , waiting for getting merge.

Interested

@pratik9818 your PR for your existing issue will need to be merged before we assign another issue to you. This ensures a fair approach and gives everyone an opportunity to contribute.
In any event, as Amanda mentioned (@jatingodnani please also take note of this) - this issue cannot be assigned until whoever is looking to work on it explains their approach to resolving it.

@SaraJaoude, I found the bug in the code. We are rendering two things: 'users' and 'visibleUsers.' 'Users' contains an array of updated users, while 'visibleUsers' contains an array of search results due to a query from the 'users' array. 'VisibleUsers' takes some time to render, so I added a loading indicator. We don't need to render updated users every time; we only show updated users if the input box is empty.

@SaraJaoude, I found the bug in the code. We are rendering two things: 'users' and 'visibleUsers.' 'Users' contains an array of updated users, while 'visibleUsers' contains an array of search results due to a query from the 'users' array. 'VisibleUsers' takes some time to render, so I added a loading indicator. We don't need to render updated users every time; we only show updated users if the input box is empty.

Thank you for explaining the approach you would like to take. I will assign this to you to begin work on the proposed solution.

Unassigning for this reason: https://github.com/EddieHubCommunity/BioDrop/pull/9788#issuecomment-1968536083

I see this is no longer assigned. Noticed it does not have the [good first issue] label, but still has 3 or less points. I managed to setup the env locally and I would like to work on it by doing the following:

create a reusable Loading component under components folder
keep the loading flag as a state and set it to true/false inside the fetchUsers method (true when starts, false when ends with success/error)
when the loading is true render the reusable Loading component, otherwise render the actual users list

Extra:

maybe add some loading delay support just not to have the loading spinner flickering if the loading time is very short
get rid of the extra logic of rendering the list with {users.length < usersPerPage && and {users.length > usersPerPage && since const visibleUsers = users.slice(indexOfFirstUser, indexOfLastUser); will always return the visible users in range even if the indexOfLastUser is bigger than the actual users array

@stefanzmf  Assigning the issue to you

Opened the PR. Please let me know if there are further steps that needs to be done from my side. Thank you

Pull Request merged


The button to delete questions and comments  by selection doesn't work

Describe the issue/bug:
When we select questions or comments to delete by selection, on the page to view user content, when clicking on the 2nd button "Delete", it doesn't work, does nothing. It happenned after the v7.19.1.
To Reproduce:

Go to user profile
Go to "view user content"
In the "Comments" or "Questions" tab, select questions or comments to delete
Click on the 2nd "Delete" button

A screenshoot: https://prnt.sc/wgezec
My Brainly profile: brainly.com.br/app/profile/16367094

Additional: the question selection eliminator stopped working after version v7.19.2.

Additional: the question selection eliminator stopped working after version v7.19.2.


Add support  for CURRENT_TIMESTAMP/NOW equivalent in the filters

As discussed in #192, it will be nice to have a date constant that can be used as part of  an api rule/filter.
Suggestions for name: @now, @time, @current_time, etc.

Implemented in master. Will be available with the next feature release (v0.5.0).


Configure logging via TOML config

Description

Now you can fully configure logging from TOML-config.
Option no_configure_logging disable any logging configuring.
Separate logging configuration from other code for better maintainability.
Also I update Apache copyright notice from my previous PR. Cindicator's leader recommend to use "Vote Inc." company name in copyrights.

Motivation and Context
Q. Why do you use same file for Luigi's config and logging config?
A. Because this is all project config. Also in future Pull Requests I plan to integrate in Luigi configs inheritance which allow also use separate file for logging, local settings or something else.
Q. Why do you do this? We already have support for logging.conf.
A. Because logging via ini-file is ugly, since ini format doesn't support inherit dicts. But TOML (and YAML. JSON and other modern formats) can do it.
Q. Why do you change no_configure_logging behavior?
A. Option no_configure_logging useful when you're configure logging in project bypass to Luigi. For example, when you want to use JSON or YAML configs for it. This is intuitive behavior for option with this name.
Q. Why do you change code structure?
A. Old code structure is hard to test, explore and extend. Now it's beautiful :)
Have you tested this? If so, how?

I have included unit tests.
Don't trust me! I update example repository, and you can play with new logging configuring yourself.

And...
Thank you for best pipeline library. And best music streaming service too. All this things is amazing!

Sorry for many commits. This is going from my previous PR, because you squash all my changes instead of merge.

I have some failed tests. I set up WIP status for project while I not find source of this troubles.

Oh and could you update the PR Title to be more clear?

You're right, fixed.

I'm in favor of this. I'll have to review another time though. :)

@Tarrasch, how is your review? ^^

Added some @Tarrasch's improvements. Other requests commented

CI is OK now. What about this PR? :)

I'm good here.
Have you been using this feature in production? All is good for your use?

Now I'm in another team in the same company. I'll get actual feedback on this feature by next few days :)

Congrats on your move @orsinium ! Appreciate the feedback

I got feedback and have fixed example based on this:

disable_existing_loggers should be false for saving already existing loggers.
dot in luigi.server logging means inheritance so we can just configure only luigi logger.

I think, it will be better change luigi-interface to luigi.interface, but I don't do it in this PR because it can broke logging in existing projects.
So. now it is ready, I hope :)

@dlstadther, @Tarrasch, what are we going to do with this Pull Request?

I just looked back through this PR and I think it LGTM. Given @Tarrasch had some changes requested, I'll wait for him to give his approval that his comments were addressed.
Thanks for continuing to follow up @orsinium !

Yeah, of course, we call InterfaceLogging instead of:
https://github.com/spotify/luigi/blob/235edcccaa207b3c03c727521a95429cc736b568/luigi/interface.py#L144


Add Cake.Frosting build project to build/test and package

This adds a new Cake.Frosting project, which build/tests and packages Textrude + -Interactive.
It uses Spectre.Console to make the Build output a bit fancier. Warning highlighting and prefixing is only done when run locally to prevent weird issues regarding automatic text overflow
One small "bug" is that when downloading the generated ZIP, it is zipped again resulting in a ZIP in a ZIP. As stated in https://github.com/actions/upload-artifact#zipped-artifact-downloads there is currently no way to change that
I opted to use Cake.Frosting instead of the one Cake file approach because:

Better IntelliSense, as it is a normal C# console app.
Allows for better organization

This looks really cool - thanks so much Martin.  I was checking in your fork periodically so appreciate how much work you put into this!


ci: fix build steps not running on main branch pipeline runs

When the pipeline runs after a merge to main, the build steps were being skipped. Since it's quite difficult in ADO to get variables from another pipeline, it's better to just run all steps when merging to main I think.

:tada: This PR is included in version 1.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Paths that include the package name causes conflicts

WARNING in omit-tilde-webpack-plugin files should use ~ to refer to modules: change "spin.js" -> "~spin.js"
where the path is:
App_Root/bower_components/spin.js/spin.js
It would appear that the directory being 'spin.js' is causing issues, incorrectly suggesting that a tilde needs to be added.

Logging candidate at line 81:
{
  path: '/Applications/XAMPP/xamppfiles/htdocs/deployment-tool/bower_components/spin.js',
  request: './spin.js',
  query: null,
  directory: false
}

Thanks @piepieonline
Could you please try installing<EMAIL_ADDRESS>and let me know if it solves the problem.

closed by #6


Add *.tableau.com as a trusted site

In Winter '21, by default the viz component is blocked from Communities because of a CSP issue.

We need to add public.tableau.com as a CSP trusted site to fix that.
This can be done automatically by including this metadata file in the project/package.

This metadata update can be deployed without risk on Summer '20.

Updated PR to include *.tableau.com domains.

@illonage I'll let you merge this PR at your convenience.

@illonage @pozil  - thanks for the heads up about this change. If I understand correctly, we should add a note about the CSP issue for users of the Winter '21 release and explain how the Tableau Lightning web component handles it (Community pages). Sounds like this is a non-issue for < Winter '21?

@d45 Yes it is not an issue for users < Winter 21 and it is not going to be an issue for Tableau Online and Public users. I opened a TFS item (1179594) to track it

I need to read this and understand why we didn't hit this in our testing. Clearly some customers have turned on this level of CSP while others have not. Should we be adding public automatically even though customers might not be using Public. I am fine doing it ... I just want to understand

Talked to Gig to get context on the upcoming SF change.