國立臺灣大學電機資訊學院電信工程學研究所 碩士論文

### Graduate Institute of Communication Engineering College of Electrical Engineering and Computer Science

### National Taiwan University Master Thesis

周全系統參數下最大化廣義分頻多工系統之頻譜效率 Maximization of Spectral Efficiency of Generalized

Frequency Division Multiplexing Systems Under Performance Constraints That Considers Comprehensive

System Parameters

羅政捷 Cheng-Chieh Lo

指導教授：蘇柏青 博士 Advisor: Borching Su, Ph.D.

中華民國 111 年 11 月 November, 2022

## 摘要

近年來，由於廣義分頻多工（generalized frequency division multi- plexing, GFDM）系統具有低頻帶外輻射（out-of-band emission, OOBE）、 低延遲、高頻譜效率（spectral efficiency）的特性，因此被推出來做為 取代正交分頻多工（orthogonal frequency division multiplexing, OFDM）

系統的候選人之一。然而，廣義分頻多工系統會因為具有非么正的傳 輸矩陣（transmitter matrix）而造成雜訊放大（noise enhancement）和 接收端在均方誤差（mean square error, MSE）以及符元錯誤率（symbol error rate, SER）上有性能下降的情形。因此，過去多項研究探討了廣 義分頻多工系統在最小化頻帶外輻射並維持良好頻帶內性能的原型濾 波器（prototype filter）設計最佳化演算法。不過，這些最佳化方法沒 有把副載波（subcarrier）大小、副符碼（subsymbol）大小、頻譜資源 分配方法、循環前綴（cyclic prefix, CP）長度納為設計參數，而這些 參數會影響到廣義分頻多工系統的頻譜效率。此外，一個有彈性的取 樣頻率（sample rate）也會影響系統的頻譜效率，但前人並未對此做過 研究。

在本研究中，我們設計了一道綜合參數的最佳化問題，在符合性 能限制的條件下最大化廣義分頻多工系統的頻譜效率。我們提出了 多項演算法來解決這道混合整數規劃問題（mixed integer programming problem）。另外，我們還應用了一些方法使提出的演算法更有效率。

模擬結果顯示，我們提出的設計方法能夠比前人的方法達到更高的頻 譜效率。

關鍵詞：廣義分頻多工，頻譜效率，混合整數規劃問題

**Abstract**

In recent years, generalized frequency division multiplexing (GFDM) sys- tems have been promoted as an alternative to orthogonal frequency divi- sion multiplexing (OFDM) systems due to their low out-of-band emission (OOBE), low latency, and high spectral efficiency. However, GFDM systems can suffer from noise enhancement and in-band performance degradation in receiver mean square error (MSE) and symbol error rate (SER) from having non-unitary transmitter matrices. Therefore, filter optimization algorithms for GFDM to minimize OOBE while maintaining good in-band performance had been studied in the past. Nevertheless, these optimization methods didn’t take subcarrier and subsymbol sizes, spectral resource allocation method, and CP length as optimization variables, and these factors have impacts on GFDM systems’ spectral efficiency. Furthermore, the effect of flexible sample rates has not yet been studied, which can also sway the systems’ spectral efficiency.

In this research, a joint optimization problem to maximize spectral effi- ciency for GFDM systems under performance constraints is introduced. The proposed method is presented to solve this mixed integer programming prob- lem by dissecting it into simpler problems, and heuristics specific to the prob- lems were incorporated to make the proposed method more time-efficient.

Simulation results show that the proposed method is able to achieve higher spectral efficiency for GFDM systems compared to existing GFDM optimiza- tion methods.

**Keywords: generalized frequency division multiplexing (GFDM), spec-**
tral efficiency, mixed integer programming problem

**Contents**

口試委員會審定書 **iii**

摘要 **v**

**Abstract** **vi**

**Contents** **vii**

**List of Figures** **ix**

**List of Tables** **xi**

**1** **Introduction** **1**

1.1 Introduction . . . 2

1.2 Notation . . . 3

**2** **System Model** **4**
2.1 GFDM System Model . . . 5

2.2 Power Spectral Density (PSD) of Analog GFDM Signals . . . 7

2.3 Spectrum Emission Mask (SEM) . . . 8

2.4 Noise Enhancement Factor (NEF) . . . 8

**3** **Problem Formulation** **9**
3.1 Problem Statement . . . 10

3.2 Proposed Method . . . 11

3.2.1 Solve Problem 1 . . . 11

3.2.2 Solve Problem 2 . . . 12

3.2.3 Solve Problem 3 . . . 15

3.2.4 Solve Problem 4 . . . 17

3.3 Speed Up the Proposed Method . . . 18

3.3.1 Speed Up Algorithm 2 . . . 19

3.3.2 Speed Up Algorithm 3 . . . 19

**4** **Simulation Results** **22**
4.1 Proposed Method vs. Existing Methods . . . 23

4.1.1 Parameter Settings . . . 23

4.1.2 Simulation Results . . . 24

4.2 Additional Simulations . . . 25

4.2.1 Parameter Settings . . . 25

4.2.2 Simulation Results . . . 26

4.2.3 Lower Sample Rate Beyond The Channel Bandwidth . . . 28

**5** **Conclusion and Future Work** **33**
5.1 Conclusion . . . 34

5.2 Future Work . . . 34

**Appendix A Lemma 1 Proof** **35**

**Appendix B Definition of Function update_f_s_LB(***·)* **36**

**Appendix C Equivalency of Problem 4 (3.6) and Problem 4.1 (3.8)** **37**

**Appendix D Obtain g from S = gg**^{H}**38**

**Bibliography** **39**

**List of Figures**

2.1 Block diagram of GFDM transceivers. . . 5

2.2 *Resource allocation for K = 5, M = 7, K*_{on}*= 3, M*_{on}*= 5,K =*
**{0, 2, 4}, and M = {0, 2, 3, 5, 6}, where reshape(v, K, M) is a K × M****matrix whose (k, m)-th element is [v]***k+mK*,*∀ 0 ≤ k < K, 0 ≤ m < M. .* 5

3.1 Contiguous subcarrier resources are used to fill in the SEM passband region. 13 3.2 PSD of used resources, PSDs of unused subsymbols, and the prototype filter for using contiguous subsymbols. Unspecified parameter settings follow that of case 3 in section 4.2. . . 13

3.3 PSD of used resources, PSDs of unused subsymbols, and the prototype fil- ter for using non-contiguous subsymbols. Unspecified parameter settings follow that of case 3 in section 4.2. . . 13

3.4 Overview of the proposed algorithms to solve problem 1 (3.2). . . 19

4.1 PSDs of case 1 and the first simulation of Chen et al. [6]. . . 25

4.2 PSDs of case 2 and the first simulation of Lin et al. [7]. . . 25

4.3 Adding CP to the top solution of simulation case 3 resulting a PSD with larger sidelobes. . . 27

4.4 *R (10*^{6} *symbol/s/Hz) of different K and M for case 3. Infeasible results*
are labeled ”n/a”. . . 29

4.5 **PSD and g for the top three solutions of case 3. . . .** 29

4.6 *R (10*^{6} *symbol/s/Hz) of different K and M for case 4. Infeasible results*
are labeled ”n/a”. . . 30

4.7 **PSD and g for the top 3 solutions of case 4. . . .** 30

4.8 *R (10*^{6} *symbol/s/Hz) of different K and M for case 5. Infeasible results*
are labeled ”n/a”. . . 31
4.9 **PSD and g for the top three solutions of case 5. . . .** 31
*4.10 R (10*^{6} *symbol/s/Hz) of different K and M for case 6. Infeasible results*

are labeled ”n/a”. . . 32
**4.11 PSD and g for the top three solutions of case 6. . . .** 32

**List of Tables**

4.1 Parameter settings for cases 1-2. . . 24

4.2 Optimal values for cases 1-2 and the cases from existing methods. . . 25

4.3 Parameter settings for cases 3-6. . . 26

4.4 Optimal values for the top 3 solutions of case 3. . . 29

4.5 Optimal values for the top 3 solutions of case 4. . . 30

4.6 Optimal values for the top 3 solutions of case 5. . . 31

4.7 Optimal values for the top 3 solutions of case 6. . . 32

**Chapter 1**

**Introduction**

**1.1** **Introduction**

Only a finite portion of the total electromagnetic spectrum supports radio communication, so it is crucial to use this precious resource as efficiently as possible [1]. One impor- tant measure is spectral efficiency, which is the amount of data bandwidth that a specific technology can extract from a certain amount of radio spectrum [1].

Orthogonal frequency division multiplexing (OFDM) has been the dominant technol- ogy for broadband multicarrier communications mainly because of its robustness against multipath channels and easy implementation based on Fast Fourier Transform (FFT) al- gorithms [2]. However, OFDM signals with one cyclic prefix (CP) per symbol would present a prohibitively low spectral efficiency [2]. The low spectrum efficiency due to CP insertion is also a problem for its applications [2]. Additionally, the high out-of-band emission (OOBE) of OFDM poses a challenge to avoid harmful interference [2]. There- fore, generalized frequency division multiplexing (GFDM) [3] has been promoted as an alternative to OFDM. It has been extensively studied to address the requirements of low latency and low OOBE [2]. Also, GFDM has flexible time-frequency structures to adapt to various application scenarios, such as multiple access and cognitive radios [2]. With low OOBE, GFDM does not require wide guard bands between adjacent bands as OFDM does, so it may achieve a higher spectral efficiency [4].

Since the GFDM transmitter matrix is in general non-unitary, GFDM can suffer from noise enhancement and in-band performance degradation in receiver mean square error (MSE) and symbol error rate (SER) compared to that achieved through OFDM [5][6].

In the research from Chen et al. (2017) [6], a filter optimization algorithm that mini- mizes OOBE while maintaining good in-band performance is proposed for GFDM. Under the same spectral efficiency, optimized filters perform the best in terms of both OOBE and SER compared to OFDM and prototype filters existing in the literature [6]. In the research from Lin et al. (2018) [7], a prototype transmit filter design method along with an idea of spectral regrowth prediction (SRP) is proposed for GFDM. The resulting OOBE can meet any given spectrum emission mask (SEM) with maximal spectral resource utilization in the GFDM system [7].

The aforementioned GFDM optimization methods [6][7] have predetermined subcar- rier and subsymbol sizes, spectral resource allocation methods, and CP lengths. These factors have impacts on GFDM signals’ power spectral density (PSD) and the optimiza- tion algorithms, influencing the system’s spectral efficiency. Moreover, a flexible sample rate can also affect the system’s spectral efficiency, so we took it into account in designing our optimization problem to maximize spectral efficiency.

In this paper, we formulate a joint optimization problem to maximize the spectral effi- ciency of GFDM systems under performance constraints, which considers subcarrier and subsymbol sizes, spectral resource allocation, prototype filter coefficient, CP length, and sample rate as optimization variables. Also, we propose several algorithms to solve this mixed integer joint optimization problem.

**1.2** **Notation**

Matrices, column vectors, and scalars are denoted by boldfaced upper case letters, bold-
**faced lower case letters, and italic letters, respectively, e.g., X, x, and x. [X]*** _{i,j}* for any

**matrix X denotes the entry of the i-th row and the j-th column of X, and [x]***n*denotes

**the n-th element of the vector x. Superscripts (**·)*, (*

^{∗}*·)*

*, (*

^{−1}*·)*

*, (*

^{T}*·)*

*, and (*

^{H}*·)*

*denote conjugate, inverse, transpose, transpose-conjugate, and Hadamard inverse (defined by*

^{◦−1}**[X**

*]*

^{◦−1}

_{m,n}**= [X**

*]*

^{−1}

_{m,n}*,∀ m, n), respectively. A set is denoted by calligraphic font, e.g.,*

*2*

**A. For any set A, we denote its cardinality by |A|. Given a matrix X, ∥X∥****, tr(X), and**

**rank(X) denotes its Frobenius norm, trace, and rank.**

*⟨·⟩*

*D*

*denotes modulo D, and E[·]*

denotes the expectation operator.Z, Z*>0**,*Z* _{≥0}*andR

*>0*denote the set of integers, positive integers, non-negative integers, and positive real numbers. C

*andH*

^{D}*+ denote the set of*

^{D}*D*

**×1 complex vectors and D×D Hermitian positive semidefinite matrices. A⊗B denotes****the Kronecker product of matrices A and B. Throughout the paper, we adopt zero-based**indexing.

**Chapter 2**

**System Model**

Figure 2.1: Block diagram of GFDM transceivers.

*Figure 2.2: Resource allocation for K = 5, M = 7, K**on* *= 3, M**on* *= 5,K = {0, 2, 4},*
and**M = {0, 2, 3, 5, 6}, where reshape(v, K, M) is a K × M matrix whose (k, m)-th****element is [v]*** _{k+mK}*,

*∀ 0 ≤ k < K, 0 ≤ m < M.*

**2.1** **GFDM System Model**

GFDM is a block-based modulation scheme as shown in Figure 2.1 [2]. Each GFDM
*block has D = KM spectral resources, which can be decomposed into K subcarriers and*
*M subsymbols. To meet performance constraints, not all spectral resources are used to*
transmit data. We perform resource allocation to map data symbols to specific resource
indices*K and M, as shown in Figure 2.2, where K ⊆ {0, 1, . . . , K − 1} is the set of*
used subcarrier indices, *M ⊆ {0, 1, . . . , M − 1} is the set of used subsymbol indices,*
*and unused resources are filled with 0. K** _{on}* =

*|K| is the number of used subcarriers,*

*M*

*=*

_{on}*|M| is the number of used subsymbols, N = K*

*on*

*M*

*is the number of used resources, and*

_{on}

^{K}

^{on}

_{KM}

^{M}*=*

^{on}

^{N}

_{D}*is the utility rate. The N*

*× 1 complex data symbol vector in*

**the l-th block transmission s[l] can be expressed as****s[l] = [s***0,0**[l], . . . , s*_{K}_{on}_{−1,0}*[l], s*_{0,1}*[l], . . . , s*_{K}_{on}_{−1,1}*[l], . . . , s*_{K}_{on}_{−1,M}_{on}_{−1}*[l]]*^{T}*.* (2.1)

*Let s*_{k,m}*[l] be zero-mean, independent, and identically distributed with symbol energy E** _{s}*,

*we obtain the D× 1 resource allocated data symbol vector in the l-th block transmission*

**d[l] by mapping s[l] to the used resources. It can be expressed as**

**d[l] = [d**_{0,0}*[l], . . . , d*_{K}_{−1,0}*[l], d*_{0,1}*[l], . . . , d*_{K}_{−1,1}*[l], . . . , d*_{K}_{−1,M−1}*[l]]*^{T}*,* (2.2)

*where d**k,m**[l] is the resource allocated data symbol on the k-th subcarrier and m-th sub-*
*symbol in the l-th block. Each resource allocated data symbol d*_{k,m}*[l] is transmitted with*
**its corresponding pulse shape g**_{k,m}*, whose n-th entry is*

**[g*** _{k,m}*]

_{n}**= [g]**

_{⟨n−mK⟩}

_{D}*· e*

^{j2π}

^{K}

^{k}

^{n}*, n = 0, 1, . . . , D− 1,*(2.3)

*where the D × 1 complex vector g is the prototype filter [2]. The D × D complex GFDM*
transmitter matrix can be modeled as [2]

**A = [g**_{0,0}**, . . . , g**_{K}_{−1,0}**, g**_{0,1}**, . . . , g**_{K}_{−1,1}**, . . . , g**_{K}_{−1,M−1}*].* (2.4)

**The transmit signal x[l] can then be obtained by x[l] = Ad[l]. Due to channel delay***spread, a CP of sample length N** _{CP}* is added to prevent intersymbol interference (ISI).

*After parallel-to-serial conversion (P/S), the digital baseband transmit signal x[n] is then*
*passes through a digital-to-analog converter (D/C) with sampling rate T** _{s}*and a pulse shap-

*ing filter p*

_{1}

*(t) to produce the continuous-time baseband signal*

*x(t) =*

∑*∞*
*n=**−∞*

∑*∞*
*l=**−∞*

∑

*k**∈K*

∑

*m∈M*

*d*_{k,m}*[l]g*_{m}*[n− lD*^{′}*]e*^{j2πk(n}^{K}^{−lD′)}*· p*1*(t− nT**s**),* (2.5)

*where D*^{′}*= D + N** _{CP}* is the number of time-domain samples in each block transmission,
and

*g*_{m}*[n] =*

**[g]**_{⟨n−mK−N}_{CP}_{⟩}_{D}*, n = 0, 1, . . . , D*^{′}*− 1*

0 *, otherwise*

*.* (2.6)

After passing the baseband signal through a wireless channel, the received signal can be
*modeled as y(t) = c(t)∗x(t)+w(t), where c(t) is the channel impulse response and w(t)*
*is a zero-mean complex additive white Gaussian noise (AWGN) with variance N*_{0}. By

*processing y(t) with a receive filter p*2*(t) and an analog-to-digital converter (C/D) with*
*sampling rate T*_{s}*, we get the discrete-time received signal, y[n]. After serial-to-parallel*
**(S/P) conversion and CP removal, we get the received sample vector y[l]. It can be shown****[2] that y[l] = Cx[l] + w[l], where C is a D**× D circulant matrix with its first column*be [c(0), c(1), . . . , c(N*_{c}*), 0, . . . , 0]*^{T}*, N*_{c}*≤ N**CP** , and w[l] is the noise vector. The D× 1*
received resource allocated data symbol vector ˆ

*(ZF) equalizer so that ˆ*

**d[l] is obtained by applying a zero-forcing**

**d[l] = B**

_{ZF}

**y[l], where B**

_{ZF}**= A**

^{−1}**C**

*[2]. Finally, we obtained the*

^{−1}*N*

**× 1 received data symbol vector ˆs[l] by unmapping ˆd[l] from the used resource indices.****2.2** **Power Spectral Density (PSD) of Analog GFDM Sig-** **nals**

The PSD of analog GFDM signals (2.5) can be derived as [8]

*S*_{a}*(f, f*_{s}**, g, N**_{CP}*,K, M) =* *E*_{s}*f*_{s}*|P*1*(f )|*^{2}
*D*^{′}

∑

*k**∈K*

∑

*m∈M*

*|G**m*

(*f*
*f*_{s}*−* *k*

*K*
)

*|*^{2}*,* (2.7)

*where f** _{s}* =

_{T}^{1}

*s* *is the sample rate, P*_{1}*(f ) is the frequency response of the pulse shaping*
*filter p*_{1}*(t), and*

*G*_{m}*(e** ^{jw}*) =

∑*∞*
*n=−∞*

*g*_{m}*[n]e** ^{−jwn}* (2.8)

*is the discrete-time Fourier transform (DTFT) of g*_{m}*[n]. G*_{m}*(e** ^{jw}*) can also be expressed as

*G*

_{m}*(e*

^{jw}**) = b**

^{H}**g**

*m*

**, where b = [1, e**

^{jw}*, e*

^{j2w}*, . . . , e*

^{j(D}

^{′}*]*

^{−1)w}

^{T}**and g**

_{m}*= [g*

_{m}*[0], g*

_{m}*[1], . . . , g*

_{m}*[D*

^{′}*−*1]]

^{T}**. b performs frequency modulation, and g**

_{m}**= P**

_{m}**g, where**

**P***m* =

**0**_{N}_{CP}_{×(D−N}_{CP}_{)} **I**_{N}_{CP}**I**_{D}

**0**_{1}* _{×(D−1)}* 1

**I**

*D*

*−1*

**0**

*(D*

*−1)×1*

*−mK*

(2.9)

*is a D*^{′}*× D matrix that performs circular shift and CP insertion.*

With some derivations, we can further obtain that

*|G**m*

(*f*
*f*_{s}*−* *k*

*K*
)

*|*^{2} **= tr(ff**^{H}**gg**^{H}* ) = tr(FS),* (2.10)

**where f = P**^{H}_{m}**b**_{k}**, b**_{k}*= [1, e** ^{j2π}*
(

*f*

*fs**−**K*^{k}

)
*, e*^{j4π}

(*f*
*fs**−**K*^{k}

)

*, . . . , e*^{j2(D}^{′}* ^{−1)π}*
(

*f*

*fs**−**K*^{k}

)

]^{T}**, F =**
**ff**^{H}**, and S = gg*** ^{H}*.

**The PSD of analog GFDM signals can then be expressed as a function of S**

*S*_{a}*(f, f*_{s}**, S, N**_{CP}*,K, M) =* *E*_{s}*f*_{s}*|P*1*(f )|*^{2}
*D*^{′}

∑

*k**∈K*

∑

*m**∈M*

* tr(FS).* (2.11)

**2.3** **Spectrum Emission Mask (SEM)**

SEM is a relative measurement of the out-of-channel emissions to the in-channel power.

The SEM measurement is used to measure the excess emissions that would interfere with
*other channels or with other systems. SEM(f ) denotes the SEM, and its channel band-*
*width C is provided alongside the SEM in networking specifications.*

**2.4** **Noise Enhancement Factor (NEF)**

The NEF of the GFDM ZF receiver can be derived as [8]

* ξ(g) =* 1

*D*

^{2}tr([

**(W***M* **⊗ I***K***)gg**^{H}**(W***M* **⊗ I***K*)* ^{H}*]

*)*

_{◦−1}*.* (2.12)

**It is proven in [8] that ξ(g)*** ≥ 1, and when ξ(g) = 1, the GFDM transmitter matrix*
(2.4) is known to be unitary [8]. A unitary transmitter matrix (2.4) implies equal noise
enhancement for each subcarrier and subsymbol [8].

**Chapter 3**

**Problem Formulation**

**3.1** **Problem Statement**

The goal of this research is to design a method to solve a joint optimization problem to
maximize the spectral efficiency of GFDM systems under performance constraints, in-
cluding noise enhancement penalty prevention, SEM requirement, ISI prevention, radio
subframe duration limit, and sample rate range limit. Optimization variables include sub-
*carrier size K, subsymbol size M , resource allocation methodK and M, prototype filter*
**g, CP length N***CP**, and sample rate f*_{s}*. The spectral efficiency R is defined as*

*R(symbol/s/Hz) =*

( *K*_{on}*M*_{on}*KM + N*_{CP}*· f**s*

)*÷ C.* (3.1)

*Let K, M, E*_{s}*, P*_{1}*(f ), SEM(f ), C, root mean square (RMS) delay spread τ*_{RMS}, sample
*rate upper bound f**s*UB*, and radio subframe duration τ*subframebe given. The problem can be
formulated as problem 1

*K,M,**K,M,N*max*CP**,f*_{s}**,g**

( *K**on**M**on*

*KM + N*_{CP}*· f**s*

)*÷ C* (3.2a)

subject to **∥g∥**^{2}2= 1 (3.2b)

**ξ(g)***≤ 1* (3.2c)

*S**a**(f, f**s***, g, N***CP**, K, M,**K, M) ≤ SEM(f) ∀f* (3.2d)

*N*_{CP}*> τ*_{RMS}*· f**s* (3.2e)

*KM + N**CP* *≤ τ*subframe*· f**s* (3.2f)

*f**s**≤ f**s*_{UB} (3.2g)

*Optimization variables K and M are in the set*Z*>0*,*K and M are themselves sets that*
are subsets ofZ_{≥0}*. N** _{CP}* is in the setZ

_{≥0}*, f*

*is in the setR*

_{s}*>0*

**, and g is in the set**C

*. The*

^{D}*objective function (3.2a) is to maximize the GFDM system’s spectral efficiency R (3.1).*

The constraint (3.2b) is to normalize the prototype filter energy. The constraint (3.2c) guarantees that the GFDM modulation matrix (2.4) is unitary so that there is no additional noise enhancement at the ZF receiver [8]. The constraint (3.2d) warrants the PSD (2.7) meets the given SEM requirement. The constraint (3.2e) prevents ISI by making sure that the CP duration is longer than the RMS delay spread [9]. Constraint (3.2f) puts a limit on GFDM frame size to guarantee the frame duration doesn’t exceed the radio subframe duration set by telecommunication standards. Lastly, using high sample rates can result in high power consumption and performance degradation in analog-to-digital converters

[10][11], so we introduce constraint (3.2f) to set a sample rate upper bound.

**3.2** **Proposed Method**

Problem 1 (3.2) is a mixed integer programming problem that is difficult to analyze. Our approach to solving problem 1 (3.2) is to recursively dissect it into smaller problems. If the smaller problem can be turned into a convex problem, we can solve it using convex optimization methods [12]. If the smaller problem is still a mixed integer programming problem, we can make it more accessible by solving some of its non-convex variables and constraints. We repeat this procedure until it produces a convex problem.

**3.2.1** **Solve Problem 1**

*To simplify problem 1 (3.2), first, we want to tackle integer variables K and M because*
**K, M, and g are dependent on them. The upper bound of the product KM is the largest***integer that is smaller than the product τ*subframe*· f**s*UB to satisfy constraint (3.2f)

*KM* *≤ τ*subframe*· f**s*UB*,∀ K, M.* (3.3)

*Those cases with K = 1 or M = 1 are excluded to preserve the time-frequency structure of*
GFDM. We can then get all candidate solutions of problem 1 (3.2) by brute-force searching
*all valid K and M combinations. The candidate with the highest spectral efficiency R (3.1)*
is the solution. This algorithm is called algorithm 1, and problem 1 (3.2) can be broken
down into algorithm 1 and problem 2, where problem 2 can be expressed as

*K,M,N*max*CP**,f**s***,g**

( *K*_{on}*M*_{on}*KM + N**CP* *· f**s*

)*÷ C* (3.4a)

subject to **∥g∥**^{2}2= 1 (3.4b)

**ξ(g)***≤ 1* (3.4c)

*S**a**(f, f**s***, g, N***CP**,**K, M) ≤ SEM(f) ∀f* (3.4d)

*N**CP* *> τ*_{RMS}*· f**s* (3.4e)

*KM + N**CP* *≤ τ*subframe*· f**s* (3.4f)

*f*_{s}*≤ f**s*UB (3.4g)

**Algorithm 1: Solve**problem 1 by solving problem 2

**Input** **: E***s**, P*_{1}*(f ), C, SEM(f ), τ*_{RMS}*, τ*_{subframe}*, f*_{s}_{UB}.
**Output** **: R, K, M,**K, M, N*CP**, f*_{s}**, g.**

**1** *R*max*← 0*

**2** **for K*** ∈ {n ∈ Z|n ≥ 2}, M ∈ {n ∈ Z|n ≥ 2} that KM ≤ τ*subframe

*· f*

*s*UB

**do**

**3** *[R*^{(K,M )}*,K*^{(K,M )}*,M*^{(K,M )}*, N*_{CP}^{(K,M )}*, f**s*^{(K,M )}**, g*** ^{(K,M )}*]

*← solve problem 2*

**4** **if R**^{(K,M )}*> R*_{max}**then**

**5** *R*_{max}*← R*^{(K,M )}

**6** *K*^{′}*← K, M*^{′}*← M*

**7** **if R**_{max}**> 0 then**

**8** **return R**_{max}*, K*^{′}*, M*^{′}*,K*^{(K}^{′}^{,M}^{′}^{)}*,M*^{(K}^{′}^{,M}^{′}^{)}*, N*_{CP}^{(K}^{′}^{,M}^{′}^{)}*, f**s*^{(K}^{′}^{,M}^{′}^{)}**, g**^{(K}^{′}^{,M}^{′}^{)}

**9** **else**

**10** **return infeasible**

**3.2.2** **Solve Problem 2**

Problem 2 (3.4) is a mixed integer programming problem. We want to simplify it by deal-
ing with integer set variables *K and M. To tackle variables K and M, we can again*
apply brute-force search to find the optimal values. However, brute-force searching all*K*
and*M combinations can be a time-intensive process. The following lemma and conjec-*
ture enable us to reduce the search space of problem 2 (3.4), that is, the set of*K and M*
candidate solutions.

*Lemma 1: For a GFDM system with K subcarriers, M subsymbols, sample rate of*
*f*_{s}*, CP length of N*_{CP}*, symbol energy E*_{s}*, and pulse shaping filter P*_{1}*(f ). Its analog*
* PSD is derived as (2.7), where D = KM . The PSD with prototype filter g, used sub-*
carrier indices

*K, and used subsymbol indices M, S*

*a*

*(f, f*

_{s}

**, g, N**

_{CP}*,K, M), is equal to*

**the PSD with prototype filter g**

*, used subcarrier indices*

^{′}*K*

*, and used subsymbol in- dices*

^{′}*M*

^{′}*, S*

_{a}*(f, f*

_{s}

**, g**

^{′}*, N*

_{CP}*,K*

^{′}*,M*

^{′}**), where [g**

*]*

^{′}

_{n}**= [g]**

_{⟨n−pK⟩}

_{D}*e*

^{j2πrn}

^{K}*, n = 0, 1, . . . , D,*

*K*

*=*

^{′}*{k*

^{′}*|(∃k ∈ K)[k*

*=*

^{′}*⟨k − r⟩*

*K*]

*}, M*

*=*

^{′}*{m*

^{′}*|(∃m ∈ M)[m*

*=*

^{′}*⟨m − p⟩*

*M*]

*},*

*∀ r ∈ 0, . . . , K − 1, p ∈ 0, . . . , M − 1.*

The proof of lemma 1 is shown in Appendix A. From lemma 1, we can derive that
circular shifted versions of a*{K, M} pair are considered redundant in the search space of*
problem 2 (3.4), because they can produce the same PSD.

*Conjecture 1: Assume the SEM being used has a continuous passband region, we*

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 -70

-60 -50 -40 -30 -20 -10 0 10

Figure 3.1: Contiguous subcarrier resources are used to fill in the SEM passband region.

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

-70 -60 -50 -40 -30 -20 -10 0 10

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

-70 -60 -50 -40 -30 -20 -10 0 10

0 10 20 30 40 50

real imaginary absolute

Figure 3.2: PSD of used resources, PSDs of unused subsymbols, and the prototype filter for using contiguous subsymbols. Unspecified parameter settings follow that of case 3 in section 4.2.

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

-70 -60 -50 -40 -30 -20 -10 0 10

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5

-70 -60 -50 -40 -30 -20 -10 0 10

0 10 20 30 40 50

real imaginary absolute

Figure 3.3: PSD of used resources, PSDs of unused subsymbols, and the prototype filter for using non-contiguous subsymbols. Unspecified parameter settings follow that of case 3 in section 4.2.

conjecture that the used subcarrier indices*K and the used subsymbol indices M should*
*be set to be contiguous to achieve the highest spectral efficiency R (3.1).*

In conjecture 1, if the SEM has a continuous passband region, it is intuitive to occupy
contiguous subcarrier resources*K to fill in said regions, as shown in Figure 3.1. In Figure*
3.2 and Figure 3.3, we compare using contiguous subsymbols and non-contiguous sub-
symbols on the proposed method. For the SEM being a low-pass mask, we can see that by
using contiguous subsymbols, the PSD has smaller sidelobes because the prototype filter
**g has fewer high-frequency components. Furthermore, by using contiguous subsymbols,**
even the PSDs of unused subsymbols have fewer high-frequency components. This can

result in a higher utility rate for using contiguous subsymbols.

For simplicity, we can fix the starting indices of*K and M to 0. Since used subcarriers*
are contiguous and the starting index is fixed, *K and K**on* contain the same amount of
information, the number of subcarriers being used, so we can represent*K using K**on*, their
relation can be written as *K = {0, . . . , K**on* *− 1}. With the same logic, we can also*
represent*M using M**on*, and their relation can be written as *M = {0, . . . , M**on* *− 1}.*

Instead of brute-force searching all*K and M combinations, the remaining candidate K*
and*M solutions can be represented by all valid K**on**and M** _{on}*combinations. The search
space of problem 2 (3.4) is thus reduced to

^{1}

(_{Kon}* ^{K}* )(

_{Mon}*) of its original size.*

^{M}*We can then solve problem 2 (3.4) by brute-force searching all valid K*_{on}*and M** _{on}*
combinations, and this algorithm is called algorithm 2. Problem 2 (3.4) can be broken
down into algorithm 2 and problem 3, where problem 3 can be expressed as

*N**CP*max*,f**s***,g**

( *K**on**M**on*

*KM + N**CP* *· f**s*

)*÷ C* (3.5a)

subject to **∥g∥**^{2}2= 1 (3.5b)

**ξ(g)***≤ 1* (3.5c)

*S*_{a}*(f, f*_{s}**, g, N***CP**,**K, M) ≤ SEM(f) ∀f* (3.5d)

*N**CP* *> τ*_{RMS}*· f**s* (3.5e)

*KM + N**CP* *≤ τ*subframe*· f**s* (3.5f)

*f**s**≤ f**s*UB (3.5g)

**Algorithm 2: Solve problem 2 by solving problem 3**

**Input** **: K, M, E**_{s}*, P*_{1}*(f ), C, SEM(f ), τ*_{RMS}*, τ*_{subframe}*, f*_{s}_{UB}.
**Output** **: R,**K, M, N*CP**, f**s***, g.**

**1** *R*^{′}_{max}*← 0*

**2** **for K**_{on}*∈ {n ∈ Z|1 ≤ n ≤ K}, M**on***∈ {n ∈ Z|1 ≤ n ≤ M} do**

**3** *K ← {0, . . . , K**on**− 1}, M ← {0, . . . , M**on**− 1}*

**4** *[R, N*_{CP}*, f*_{s}**, g]**← solve problem 3

**5** **if problem 3 is feasible then**

**6** **if R > R**^{′}_{max}**then**

**7** *R*^{′}_{max} *← R*

**8** *K*sol*← K, M*sol*← M, N**CP*sol *← N**CP**, f*_{s}_{sol} *← f**s***, g**_{sol} **← g**

**9** **if R**^{′}_{max}**> 0 then**

**10** **return R**^{′}_{max}*,K*sol*,M*sol*, N**CP*_{sol}*, f**s*_{sol}* , g*sol

**11**

**else**

**12** **return infeasible**

**3.2.3** **Solve Problem 3**

Problem 3 (3.5) is a mixed integer programming problem. We can simplify it by tackling
*the integer variable N*_{CP}*and the positive real variable f** _{s}*.

**An optimization problem, problem 4, is introduced to find the prototype filter g that**
satisfies constraints (3.5b)-(3.5d)

find **g** (3.6a)

subject to **∥g∥**^{2}2= 1 (3.6b)

**ξ(g)***≤ 1* (3.6c)

*S**a**(f, f**s***, g, N***CP**,**K, M) ≤ SEM(f), ∀f* (3.6d)

*Even though f**s*is a continuous variable, it is not convex in (3.5d), so we can’t solve
it using convex optimization methods. Instead, by treating the feasibility of problem 4
*(3.6) as a monotonic function of f*_{s}*, that the problem is infeasible when f** _{s}*is larger than a

*certain value, we can solve f*

*of problem 3 (3.5) by utilizing the bisection method under the radio subframe constraint (3.5f) and the sample rate upper bound constraint (3.5g).*

_{s}*Given f** _{s}*, the minimal CP length that meets the ISI prevention constraint (3.5e) can be
derived by applying the ceiling function to the right side of (3.5e)

*N**CP* =*⌈τ*RMS*· f**s**⌉.* (3.7)

Problem 3 (3.5) can thus be broken down into algorithm 3 and problem 4 (3.6).

*At the beginning of algorithm 3, we first update the sample rate lower bound f**s*LB so it
meets the subframe duration constraint (3.5f) and the sample rate upper bound constraint
(3.5g) by calling the function update_f_s_LB(*·), which is defined in Appendix B. Then,*
*let [f*_{s}_{a}*, f*_{s}_{b}*] be the starting range of the bisection method, and R*^{′′}_{max}be the highest spectral
*efficiency R (3.1) reached in an arbitrary point of the algorithm. Each iteration of the*
bisection method performs these steps:

*1. Calculate f** _{s}* =

^{f}

^{sa}

^{+f}_{2}

*, the midpoint of the interval.*

^{sb}*2. Calculate the minimum CP length N**CP* using equation (3.7) so (3.5e) is satisfied.

*3. Check if f** _{s}*fits the radio subframe constraint (3.5f). If it fits, then proceed to solve

*problem 4. Otherwise, update f**s**a*.

(a) Solve problem 4.

*(b) If problem 4 is feasible and it reaches a higher spectral efficiency R (3.1) than*
*R*^{′′}_{max}*, then updates f*_{s}_{a}*, R*^{′′}_{max} and other output variables. Otherwise, update
*f*_{s}* _{b}*.

4. Stop when ^{f}^{sb}^{−f}_{2} ^{sa}*< ϵ, where ϵ > 0 is the bisection tolerance.*

**Algorithm 3: Solve**problem 3 by solving problem 4

**Input** **: K, M,**K, M, E*s**, P*_{1}*(f ), C, SEM(f ), τ*_{RMS}*, τ*_{subframe}*, f*_{s}_{UB}*, ϵ.*

**Output** **: R, f***s**, N**CP***, g.**

**1** *f*_{s}_{LB} *← update_f_s_LB(KM, C, τ*RMS*, τ*_{subframe}*, f*_{s}_{UB})

**2** **if f**_{s}_{LB} **is not valid then**

**3** **return infeasible**

**4** *f*_{s}_{a}*← f**s*LB

**5** *f*_{s}_{b}*← f**s*UB

**6** *R*^{′′}_{max}*← 0*

**7** **repeat**

**8** *f*_{s}*←* ^{f}^{sa}^{+f}_{2} ^{sb}

**9** *N*_{CP}*← ⌈τ*RMS*· f**s**⌉*

**10** **if f***s* *≥* ^{KM +N}_{τ}_{subframe}^{CP}**then**

**11** **[R, g]***← solve problem 4*

**12** **if problem 4 is feasible then**

**13** **if R > R**^{′′}_{max}**then**

**14** *R*^{′′}_{max}*← R*

**15** *f*_{s}_{a}*← f**s*

**16** *N*_{CP}_{sol} *← N**CP**, f*_{s}_{sol} *← f**s***, g**_{sol} **← g**

**17** **else**

**18** *f*_{s}_{b}*← f**s*

**19** **else**

**20** *f*_{s}_{b}*← f**s*

**21** **else**

**22** *f*_{s}_{a}*← f**s*

**23** **until** ^{f}^{sb}^{−f}_{2} ^{sa}*< ϵ;*

**24** **if R**^{′′}_{max}**> 0 then**

**25** **return R**^{′′}_{max}*, N*_{CP}_{sol}*, f*_{s}_{sol}**, g**_{sol}

**26** **else**

**27** **return infeasible**

**3.2.4** **Solve Problem 4**

Problem 4 (3.6) is not convex. To turn it into a convex problem, we implement the method of semidefinite relaxation (SDR) [13] with a rank 1 constraint, which is used by Chen et al. [6].

**A positive semidefinite matrix S = gg*** ^{H}* is introduced. Constraint (3.6b) can be rewrit-

**ten as tr(S) = 1. The NEF (2.12) constraint (3.6c) can be rewritten as a function of S as**

**ξ(S) =**

_{D}^{1}2tr([

**(W**_{M}**⊗ I***K***)S(W**_{M}**⊗ I***K*)* ^{H}*]

*)*

_{◦−1}. The PSD constraint (3.6d) can be ex-
**pressed as a function of S as (2.11). Problem 4 (3.6) can thus be turned into an equivalent**
problem, problem 4.1, with the addition of a rank 1 constraint

find **S***∈ H** ^{D}*+ (3.8a)

subject to **tr(S) = 1** (3.8b)

**ξ(S)***≤ 1* (3.8c)

*S*_{a}*(f, f*_{s}*, N*_{CP}**, S,***K, M) ≤ SEM(f), ∀f* (3.8d)

**rank(S) = 1** (3.8e)

Problem 4 (3.6) and problem 4.1 (3.8) are equivalent problems, and the details are explicated in Appendix C. The constraint (3.8b) is affine, the constraint (3.8c) and con- straint (3.8d) are convex. However, the rank 1 constraint (3.8e) is not convex. Like Chen et al. [6], we tackle the non-convex rank 1 constraint (3.8e) by using the Dattorro iterative algorithm [14].

**S**^{(ψ)}**= Q**^{(ψ)}**Λ**^{(ψ)}**[Q*** ^{(ψ)}*]

^{H}**is the eigendecomposition of S with ordered (non-increasing)**

**eigenvalues in the ψ-th iteration, where e**

_{n}*is the eigenvector corresponds to the n-th eigen-*

*value σ*

*n*

*, n = 0, 1, . . . , D*

**− 1. V**

^{(ψ+1)}**= U**

^{(ψ)}**[U**

*]*

^{(ψ)}

^{H}**, where U**

^{(ψ)}**is Λ**

*with its first*

^{(ψ)}**column removed, and V**

^{(0)}

**= 0. V**

^{(ψ+1)}*represents the (D*

*− 1)-dimensional space or-*

**thogonal to the principal eigenvector of S**

*. The non-convex rank 1 constraint (3.8e)*

^{(ψ)}**is replaced by tr(SV**

^{(ψ)}**) in the objective function. If the column space of V**

*is the or-*

^{(ψ)}**thogonal complement of span(e**

_{0}

**), then minimizing tr(SV**

*) suppresses the eigenvalues*

^{(ψ)}

**of S that is not σ**_{0}

*. When the second largest eigenvalue σ*

_{1}

**of S is smaller than a certain**

*by iteratively solving problem 4.2, where problem 4.2 can be expressed as*

**threshold µ > 0, S can be seen as a rank 1 matrix. We can thus tackle problem 4.1 (3.8)**min

**S***∈H** ^{D}*+

**tr(SV*** ^{(ψ)}*) (3.9a)

subject to **tr(S) = 1** (3.9b)

**ξ(S)***≤ 1* (3.9c)

*S**a**(f, f**s**, N**CP***, S,***K, M) ≤ SEM(f), ∀f* (3.9d)

Problem 4.2 (3.9) is a convex problem, and CVX [15] was used to construct and solve

**the convex program. A rank 1 S can be obtained by iteratively solving (3.9), and the pro-**
**totype filter g can be derived by g =***√*

*σ*_{0}**e**0, where the derivation is detailed in Appendix
D. The algorithm is called algorithm 4.

**Algorithm 4: Dattorro iterative algorithm to solve problem 4.2**
**Input** **: K, M,**K, M, N*CP**, E*_{s}*, P*_{1}*(f ), SEM(f ), C, f*_{s}*, µ.*

**Output** **: g.**

**1** **V**^{(0)} **← 0**

**2** *ψ* *← 0*

**3** **repeat**

**4** **cvx**

**5** **variable S(D, D) hermitian nonnegative semidefinite**

**6** **minimize tr(SV*** ^{(ψ)}*)

**7** subject to

**8** **tr(S) == 1**

**9** **ξ(S)***≤ 1*

**10** *S*_{a}*(f, f*_{s}*, N*_{CP}**, S,**K, M) ≤ SEM(f)

**11** **if the problem is infeasible then**

**12** **return infeasible**

**13** **[Q**^{(ψ)}**Λ*** ^{(ψ)}*]

**← eig(S)****14** **U**^{(ψ)}**← Λ**^{(ψ)}*[:, 1 : D− 1]*

**15** **V**^{(ψ+1)}**← U**^{(ψ)}**[U*** ^{(ψ)}*]

^{H}**16** **e**0 **← Λ**^{(ψ)}*[:, 0], σ*_{0} **← Q**^{(ψ)}*[0, 0], σ*_{1} **← Q**^{(ψ)}*[1, 1]*

**17** *ψ* *← ψ + 1*

**18** * until σ*1

*< µ;*

**19** **g***← √σ*0**e**_{0}

**20** **return g**

*Using algorithms 1-4, we can solve problem 1 given E*_{s}*, P*_{1}*(f ), C, SEM(f ), τ*_{RMS}*,*
*f*_{s}_{UB}*, and τ*_{subframe}, as shown in Figure (3.4).

**3.3** **Speed Up the Proposed Method**

Brute-force search and the bisection method can be time-intensive processes. To make the proposed method more time-efficient, some heuristics specific to the problems are introduced.

Figure 3.4: Overview of the proposed algorithms to solve problem 1 (3.2).

**3.3.1** **Speed Up Algorithm 2**

*To speed up algorithm 2, let R*^{′}_{max} be the highest spectral efficiency (3.1) reached in an
arbitrary point of the algorithm, we can factor out the set of candidates with utility rate

*K**on**M**on*

*KM* lower than ^{R}_{f}^{′}^{max}^{C}

*sUB* , because those candidates can’t reach a higher spectral efficiency
*R (3.1) than R*^{′}_{max} *without f*_{s}*exceeding the sample rate upper bound f*_{s}_{UB}. Moreover,
*we can reorder the search space in the descending order of K*_{on}*M** _{on}* to execute only the

*valid candidates. Also, we set the sample rate lower bound f*

_{s}_{LB}in this step instead of in

*algorithm 3, so the algorithm can be terminated here if the sample rate lower bound f*

_{s}_{LB}is invalid without going into the subsequent algorithms. This updated algorithm is called algorithm 2.1, and it can replace algorithm 2 in the proposed method without affecting the optimal values.

**3.3.2** **Speed Up Algorithm 3**

*To speed up algorithm 3, we first try to solve problem 4.2 (3.9) with f*_{s}*= f*_{s}* _{a}*. If it is
infeasible, then we know that problem 3 (3.5) is also infeasible, eliminating doing the
bisection method. If it is feasible, we will run the bisection method to see if it can reach a

*higher spectral efficiency R (3.1). Additionally, we only have to solve problem 4.2 (3.9) if*

*the updated f*

_{s}*warrants a higher spectral efficiency R (3.1) considering its CP length. This*updated algorithm is called algorithm 3.1, and it can replace algorithm 3 in the proposed method without affecting the optimal values.

**Algorithm 2.1: Speed up algorithm 2**

**Input** **: K, M, E***s**, P*1*(f ), C, SEM(f ), τ*RMS*, τ*subframe*, f**s*UB.
**Output** **: R,**K, M, N*CP**, f*_{s}**, g.**

**1** *R*^{′}_{max}*← 0*

**2** *γ* *← 0*

**3** *f*_{s}_{LB} *← update_f_s_LB(KM, C, τ*RMS*, τ*_{subframe}*, f*_{s}_{UB})

**4** **if f**_{s}_{LB} **is invalid then**

**5** **return infeasible**

**6** **for K**_{on}*∈ {n ∈ Z|1 ≤ n ≤ K}, M**on***∈ {n ∈ Z|1 ≤ n ≤ M} in the descending****order of K**_{on}*M*_{on}**do**

**7** **if** ^{K}^{on}_{KM}^{M}^{on}*>* ^{R}_{f}^{max}^{′}^{C}

*sUB* **then**

**8** *K ← {0, . . . , K**on**− 1}, M ← {0, . . . , M**on**− 1}*

**9** **if γ****̸= 0 then**

**10** *f*_{s}_{LB} *← update_f_s_LB(KM,*_{K}_{on}^{γ}_{M}_{on}*, τ*_{RMS}*, τ*_{subframe}*, f*_{s}_{UB})

**11** **if f***s*LB **is invalid then**

**12** **break**

**13** *[R, N*_{CP}*, f*_{s}**, g]**← solve problem 3

**14** **if problem 3 is feasible then**

**15** **if R > R**^{′}_{max}**then**

**16** *R*^{′}_{max}*← R*

**17** *K*sol *← K, M*sol *← M, N**CP*sol *← N**CP**, f*_{s}_{sol} *← f**s***, g**_{sol} **← g**

**18** *γ* *← K**on**M*_{on}*f*_{s}

**19** **else**

**20** **break**

**21** **if R**^{′}_{max}**> 0 then**

**22** **return R**^{′}_{max}*,K*sol*,M*sol*, N*_{CP}_{sol}*, f*_{s}_{sol}**, g**_{sol}

**23** **else**

**24** **return infeasible**