Recently, generative adversarial network (GAN) has shown its strong ability on modeling data distribution via adversarial learning. Cross-modal GAN, which attempts to utilize the power of GAN to model the cross-modal joint distribution and to learn compatible cross-modal features, is becoming the research hotspot. However, the existing cross-modal GAN approaches typically 1) require labeled multimodal data of massive labor cost to establish cross-modal correlation; 2) utilize the vanilla GAN model that results in unstable training procedure and meaningless synthetic features; and 3) lack of extensibility for retrieving cross-modal data of new classes. In this article, we revisit the adversarial learning in existing cross-modal GAN methods and propose Joint Feature Synthesis and Embedding (JFSE), a novel method that jointly performs multimodal feature synthesis and common embedding space learning to overcome the above three shortcomings. Specifically, JFSE deploys two coupled conditional Wassertein GAN modules for the input data of two modalities, to synthesize meaningful and correlated multimodal features under the guidance of the word embeddings of class labels. Moreover, three advanced distribution alignment schemes with advanced cycle-consistency constraints are proposed to preserve the semantic compatibility and enable the knowledge transfer in the common embedding space for both the true and synthetic cross-modal features. All these add-ons in JFSE not only help to learn more effective common embedding space that effectively captures the cross-modal correlation but also facilitate to transfer knowledge to multimodal data of new classes. Extensive experiments are conducted on four widely used cross-modal datasets, and the comparisons with more than ten state-of-the-art approaches show that our JFSE method achieves remarkably accuracy improvement on both standard retrieval and the newly explored zero-shot and generalized zero-shot retrieval tasks.

@en

In this article, we address the problem of visual question generation (VQG), a challenge in which a computer is required to generate meaningful questions about an image targeting a given answer. The existing approaches typically treat the VQG task as a reversed visual question answer (VQA) task, requiring the exhaustive match among all the image regions and the given answer. To reduce the complexity, we propose an innovative answer-centric approach termed radial graph convolutional network (Radial-GCN) to focus on the relevant image regions only. Our Radial-GCN method can quickly find the core answer area in an image by matching the latent answer with the semantic labels learned from all image regions. Then, a novel sparse graph of the radial structure is naturally built to capture the associations between the core node (i.e., answer area) and peripheral nodes (i.e., other areas); the graphic attention is subsequently adopted to steer the convolutional propagation toward potentially more relevant nodes for final question generation. Extensive experiments on three benchmark data sets show the superiority of our approach compared with the reference methods. Even in the unexplored challenging zero-shot VQA task, the synthesized questions by our method remarkably boost the performance of several state-of-the-art VQA methods from 0% to over 40%. The implementation code of our proposed method and the successfully generated questions are available at https://github.com/Wangt-CN/VQG-GCN.

@en

We systematically study the indirect interaction between a magnon mode and a cavity photon mode mediated by traveling photons of a waveguide. From a general Hamiltonian, we derive the effective coupling strength between two separated modes, and obtain the theoretical expression of the system's transmission. Accordingly, we design an experimental setup consisting of a shield cavity photon mode, a microstrip line, and a magnon system to test our theoretical predictions. From measured transmission spectra, indirect interaction, as well as mode hybridization, between two modes can be observed. All experimental observations support our theoretical predictions. In this work we clarify the mechanism of traveling photon mediated interactions between two separate modes. Even without spatial mode overlap, two separated modes can still couple with each other through their correlated dissipations into a mutual traveling photon bus. This conclusion may help us understand the recently discovered dissipative coupling effect in cavity magnonics systems. Additionally, the physics and technique developed in this work may benefit us in designing new hybrid systems based on the waveguide magnonics.

@en

To probe into the mechanical behaviour of railway transition zone from the macro-meso aspects, a numerical model of transition zone is built that hybrids the Discrete Element Method (DEM) and Finite Difference Method (FDM). The DEM is utilised to simulate the ballast bed and sleeper, because it can consider the realistic ballast shapes and complex contacts between them. The FDM based on the continuum theory is utilised to simulate the track substructure according to a real structural form. Afterwards, the coupling algorithm is used to achieve the hybrid DEM-FDM simulation. The engineering practicality of this model is validated using the dynamic responses of the transition zone from a field measurement, and the macro-meso mechanical behaviour of the transition zone is analysed with or without the wedge-shape backfill. The numerical results indicate that applying the wedge-shape backfill can considerably reduce the sudden changes of track vibration and substructure surface stresses at the vicinity of the connection between the structure and embankment. Moreover, the mesoscopic results show that the acceleration vector of the soil near the rigid structure fluctuates towards the embankment and the velocity responses of track components and substructure increase gradually when the train travels from the rigid structure to the soft embankment.@en

A major challenge in matching images and text is that they have intrinsically different data distributions and feature representations. Most existing approaches are based either on embedding or classification, the first one mapping image and text instances into a common embedding space for distance measuring, and the second one regarding image-text matching as a binary classification problem. Neither of these approaches can, however, balance the matching accuracy and model complexity well. We propose a novel framework that achieves remarkable matching performance with acceptable model complexity. Specifically, in the training stage, we propose a novel Multi-modal Tensor Fusion Network (MTFN) to explicitly learn an accurate image-text similarity function with rank-based tensor fusion rather than seeking a common embedding space for each image-text instance. Then, during testing, we deploy a generic Cross-modal Re-ranking (RR) scheme for refinement without requiring additional training procedure. Extensive experiments on two datasets demonstrate that our MTFN-RR consistently achieves the state-of-the-art matching performance with much less time complexity.

@en

In this paper, we propose a novel approach to video captioning based on adversarial learning and long short-term memory (LSTM). With this solution concept, we aim at compensating for the deficiencies of LSTM-based video captioning methods that generally show potential to effectively handle temporal nature of video data when generating captions but also typically suffer from exponential error accumulation. Specifically, we adopt a standard generative adversarial network (GAN) architecture, characterized by an interplay of two competing processes: a 'generator' that generates textual sentences given the visual content of a video and a 'discriminator' that controls the accuracy of the generated sentences. The discriminator acts as an 'adversary' toward the generator, and with its controlling mechanism, it helps the generator to become more accurate. For the generator module, we take an existing video captioning concept using LSTM network. For the discriminator, we propose a novel realization specifically tuned for the video captioning problem and taking both the sentences and video features as input. This leads to our proposed LSTM-GAN system architecture, for which we show experimentally to significantly outperform the existing methods on standard public datasets.

@en

Strong noise is one of the toughest problems in the controlled-source electromagnetic (CSEM) method, which highly affects the quality of recorded data. The three main types of noise existing in CSEM data are periodic noise, Gaussian white noise, and nonperiodic noise, among which the nonperiodic noise is thought to be the most difficult to remove. We have developed a novel and effective method for removing such nonperiodic noise by formulating an inverse problem that is based on inverse discrete Fourier transform and several time windows in which only Gaussian white noise exists. These critical locations, which we call reconstruction locations, can be found by taking advantage of the continuous wavelet transform (CWT) and the temporal derivative of the scalogram generated by CWT. The coefficients of the nonperiodic noise are first estimated using the new least-squares method, and then they are subtracted from the coefficients of the raw data to produce denoised data. Together with the nonperiodic noise, we also remove Gaussian noise using the proposed method. We validate the methodology using real-world CSEM data.

@en

Cross-modal retrieval aims to enable flexible retrieval experience across different modalities (e.g., texts vs. images). The core of crossmodal retrieval research is to learn a common subspace where the items of different modalities can be directly compared to each other. In this paper, we present a novel Adversarial Cross-Modal Retrieval (ACMR) method, which seeks an effective common subspace based on adversarial learning. Adversarial learning is implemented as an interplay between two processes. The first process, a feature projector, tries to generate a modality-invariant representation in the common subspace and to confuse the other process, modality classifier, which tries to discriminate between different modalities based on the generated representation. We further impose triplet constraints on the feature projector in order to minimize the gap among the representations of all items from different modalities with same semantic labels, while maximizing the distances among semantically different images and texts. Through the joint exploitation of the above, the underlying cross-modal semantic structure of multimedia data is better preserved when this data is projected into the common subspace. Comprehensive experimental results on four widely used benchmark datasets show that the proposed ACMR method is superior in learning effective subspace representation and that it significantly outperforms the state-of-the-art cross-modal retrieval methods.

@en

Modern active distribution networks make use of intelligent switching actions to restore supply to end users after faults. This complicates the reliability analysis of such networks, as the number of possible switching actions grows exponentially with network size. This paper proposes an approximate reliability analysis method where switching actions are modelled implicitly. It can be used graphically as a model reduction method, and simulated using time-sequential or state sampling Monte Carlo methods. The method is illustrated on a simple distribution network, and reliability indices are reported both as averages and distributions. Large speedups result from the use of biased non-sequential Monte Carlo sampling - a method that is hard to combine with explicit switching models.

@en